DOCUMENT RESUME 



ED 287 858 



TM 870 559 



TITLE 

INSTITUTION 

REPORT NO 
PUB DATE 
NOTE 

PUB TYPE 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Psychological Sciences Division: 1985 Programs. 

Office of Naval Research, Washington, B.C. 

Psychological Sciences Div. 

OCNR-11487-6 

Dec 85 

189p. 

Reports - General (140) — Reference Materials - 
Bibliographies (131) 

MF01/PC08 Plus Postage. 

Adult Education; ^Cognitive Processes; *Cognitive 
Psychology; ^Decision Making; Group Behavior; Human 
Factors Engineering; Inforaiation Processing; 
Literature Reviews; Military Personnel; ^Military 
Training; *Psycholcgical Studies; Research Design; 
^Research Projects; Training 
*Office of Naval Research 



ABSTRACT 

This booklet describes research carried out under 
sponsorship of the Psychological Sciences Division of the U.S. Office 
of Naval Research during Fiscal Year 1985. The booklet is divided 
into three programmatic research areas: (1) Engineering Psychology; 
(2) Personnel and Training; and (3) Group Psychology. Each program is 
described by an overview which is followed by thematic clusters of 
related efforts. Each cluster is described by individual projects, or 
work units, which were either completed or active during 1985. Each 
report of research is about one page, and includes scientific 
objectives, approach, progress, potential applications, and funding 
source. All work unit descriptions also include a listing of the 
technical reports and archival publications produced during the year. 
Most of the programs are basic in nature, with a selected 
augmentation of applied research and exploratory development effort. 
The range is from direct military decision making applications to 
instructional theories and computer-assisted instruction. 
(Author/MDE) 



* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 
******************************************** 



ERLC 



us DCMfVTMtNTOr tOOCATION 
Offtct ol Educational Raaaarch and impfovanwnt 

EDUCATIONAL RESOURCES INFORMATION 



JlPThii documant hai bean reproduced «• 
received from the person or orgenization 
originating it 
□ Minor changes have been made to improve 
reproduction quality 




CENTER (ERIC) 



• Pointi of view or optmoni stated m thii docu- 
ment do not necessarily represent official 
OERI position or policy 



Psychological Sciences Division 
1985 Programs 




OFFICE OF NAVAL RESEARCH 
Arfington. VA 22217-5000 

December 1985 

This document is issued primarily for the information of U.S. 
Government scientific personnel and contractors. It is not 
considered part of the scientific literature and should not be 
cited as such. 

APPROVED FOR PUBLIC RELEASE: DISTRIBUTION UNLIMITED 



UNCLASSIFIED 



SECU^vlT^ CLAS$>FICATI0N0F THIS PAGE 



REPORT DOCUMENTATION PAGE 



la. REPORT SECURITY CLASSIFICATION 
UNCLASSIFIED 


lb RESTRICTIVE MARKINGS 


2a. SECURITY CLASSIFICATION AUTHORITY 
NONE 


3 DISTRIBUTION /AVAILABILITY OF REPORT ^ 

Approved Sox pt3W,tc release: 
DJstrtbutJpn PnUiajKed, 


2b DECLASSIFICATION /DOWNGRADING SCHEDULE 
NOm 


4 PERFORMING ORGANIZATION REPORT NUMBER(S} 

442-5 


5. MONITORING ORGANIZATION REPORT NUMBER(S> 


6a NAME OF PERFORMING ORGANIZATION 
Office of Naval Research 


6b OFFICE SYMBOL 
(If applicabi^) 

Code 1142 


7a NAME OF MONITORING ORGANIZATION 


6c. ADDRESS {City, SUte, ind ZIPCode) 

Psychological Sciences Division - CoJe 1142 
800 N. Qulncy Street 
Arlington, VA 22217-5000 


7b ADDRESS (0(y, State, and ZIP Code) 


8a. NAME OF FUNDING /SPONSORING 
ORGANIZATION 


8b OFFICE SYMBOL 
Of spphcabh) 


9 PROCUREMENT 'NSTRUMENT IDENTIFICATION NUMBER 


8c ADDRESS (City, Sejt«,ind2/P Code; 


10 SOURCE OF FUNDING NUMBERS 


PROGRAM 
ELEMENT NO 


PROJECT 
NO 


TASK 
NO 


WORK UNIT 
ACCESSION NO 



1 1 TITLE (Include Security Classification) 



Psychological Sciences Division 1985 Programs (U) 



12 PERSONAL AUTHOR(S) 

Office of Naval Research - Code 1142 



13a TYPE OF REPOR'' 


13b TIME COVERED 


14 DATE OF REPORT {Year, Montt), Day) 


15 PAGE COUNT 


Summary Report 


FROM 1/1/85 TO 12/1/85 


178 



16 SUPPLEMENTARY NOTATION 



17 



COSATI CODES 



FIELD 


GROUP 


SUB-GROUP 









IB SUBJECT TERMS (Continue on reverse if ryecessary and identify by block riumber) 



^9 ABSTRACT (Conffnu« on reverse if ryecessary aryd identify by block r^umber) 



20 DISTRIBUTION /AVAILABILITY OF ABSTRACT 

HUNCLASSIFIED/UNMIVHTED □ SAME AS RPT DdTIC USERS 


21 ABSTRACT SECURITY CLASSIFICATION 


22a NAME OF RESPOfiSlBLE INDIVIDUAL 

James Lester 


22b TELEPHONE (/nc/uc^ Area Code) 
(202) 696-4503 


?2c OFFICE SYMBOL 

Code 1I42CS 



DO FORM 1 473. B4 MAR 83 APR«d,t,onm*y be used unt,l exhausted SECURITY CLASSIFICATION O F THIS PAGE 

All Other editions are obsolete 



ERIC 



4 



FOREWORD 



This booklet describes research carried out under sponsorship of the 
Psychological Sciences Division of ONR during Fiscal Year 1985. The booklet 
is divided into three programmatic research areas: Engineering Psychology; 
Personnel and Training; and Group Psychology. Each program is described by an 
overview which is followed by thematic clusters of related efforts. Each 
cluster is described by individual projects, work units, which were either 
completed or active during 1985. All work unit descriptions include a lis*:ing 
of the technical reports and archival publications produced during the year. 
The report listings are not cumulative; information regarding reports published 
in previous years may be obtained from previous booklets in this series, from 
the Principal Inves tig? tors, or from this office. 

Many of the technical reports cited are available from the National 
Technical Information Service ''"'TIS), Springfield, Virginia, 22161; or the 
Defense Technical Information Center (DTIC) , Cameron Station, Building 5, 
Alexandria, Virginia, 22314. Your request will be expedited if you use the 
"AD number" which is given as part of the citation. 

This is one of reveral means by which we communicate and coordinate our 
efforts with other members of the research sponsoring and performing 
communities. We encourage your comments about any feature of this booklet or 
about the programs theoiselves. If you wish further information, please do not 
hesitate to contact members of the staff listed in the Introduction. We 
welcome your interest ir our programs and hope that you will continue to keep 
us informed of related research efforts. 



W. S. VAUGHAN, JR 
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INTRODUCTION 



Psychological Sciences Division programs are carried out under contracts 
awarded in response to unsolicited proposals. They are evaluated 
on the scientific merit of the proposed research, the facilities available for 
its conduct, the competence of the principal investigator, and relevance to 
Navy needs. The elements that shape our research program are scientific 
gaps and opportunities, and operational needs identified in Navy 
planning documents. Our overall aim is to support good science for the good 
of the Navy and the nation. 

Most of the programs are basic in nature, with a selected augmentation of 
applied research and exploratory development effort. This mix of basic 
and applied research is developed and managed by the Division staff with the 
able assistance of other ONR scientists and with helpful guidance and advice 
from representatives of various Navy and Marine Corps activities. The 
programs seek to involve innovative civilian scientists in areas of research 
relevant to Navy and Marine Corps interests, and by so doing provide new 
perspectives, new insights, and new approaches to naval manpower, personnel, 
training, equipment and system design problems. This arrangement provides 
channels for information to flow back and forth between the civilian research 
community and the naval community, each keeping the other abreast of new 
developments. The emphasis is on the creation and exploitation of a 
cumulative scientific knowledge base upon which new technologies can be 
developed to improve effectiveness of Navy and Marine Corps men and women. 

Continuous efforts are made to coordinate the Division's research program 
with other ONR contractors, with in-house Navy laboratories, and with the 
research sponsored by other services and other agencies. We work closely with 
research managers at the Navy Personnel Research and Development Center, the 
Chief of Naval ^.ducation and Training, the Naval Medical Research and 
Development Command, the Marine Corps, and the Naval Systems Commands and 
their laboratories and centers to promote the diffusion, extension, and 
eventual utilization of the knowledge obtained through the ONR contract 
research programs. 

The Psychological Sciences Division is part of the Life Sciences 

Directorate, which also includes the Biological Sciences Division. Dr. St3ven 

F. Zornetzer is Director for Life Sciences, and Commander Warren W. Schultz is 
the Deputy Director for Life Sciences. 
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The members of the staff of the Psychological Sciences DlvlsJon are listed 
below: 



Dr. W. S. Vaughan, Jr., Division Head (Acting) 
Telephone (202) 696-4505 

Engineering Psychology Programs: 

Mr. Gerald Malecki, Program Manager (Acting) 
Dr. Eugene E. Gloye* 
Dr. John J. O'Hare 
Telephone (202) 696-4290 



Group Psychology Research Programs: 

Dr. Bert T. King, Program Manager 
Dr. James T. Lester 
Telephone (202) 696-4209 



Personnel and Training Research Programs: 

Dr. Susan E. Chipman, Program Manager 
Dr. Charles E. Davis 
Dr. Michael G. Shafto 
Dr. Harold L. Hawkins 
Telephone (202) 696-4318 

*Dr. Gloye is located at the ONR Detachment, Pasadena, 1030 East 
Green Street, Pasadena, California 91106; his telephone number is 
(213) 795-5971. 



In addition to the headquarters personnel listed above, the 
following psychologists supported the program from the 
regional offices: 

Dr. Richard E. Snow 
Office of Naval Research Branch 
22 3 Old Marylebone Road 
London NWl 5TH England 
Telephone 01-409-4471 



Dr. Nicholas A. Bond 
Office of Naval Research 
Liaison Office, Far East 
APO San Francisco, CA 96503 
Telephone 813-401-8924 
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FNGINEERING PSYCHOLOGY PROGRAMS 



(CLUSTERS A-E) 



Engineering Psychology Programs are concerned with assuring effective 
performance by personnel operating and maintaining the high technology 
equipment characteristic of modern Navy and Marine Corps systems. We seek 
better fundamental understanding of human motor, perceptual and cognitive 
behavior in order to develop general guidelines for the design of compatible 
interfaces between people and machines. Our program currently 
emphasizes theory-based research leading to process-oriented models which 
explain human behavioral characteristics. 

The programs are organized into five clusters : 

A. Man-Machine System Interaction ; Objective of this cluster is to 
develop theoretical foundations of scientific knowledge about human 
limitations and capabilities in interacting with intelligent, adaptive 
computer-based systems. Current examples of research interests include 
interactive dialogue issues, resource theory for input/output modality design, 
algorithm manipulation in image processing, and cognitive compatibility issues 
in expert systems. 

B. Perception ; Objective of this cluster is to develop theories of 
perception in an information processing framework. Emphasis is on expansion 
of computational theories of human perceptual performance which account for 
both psychophysical and neurophysiological evidence. 

C. Decision Making ; Objective is to develop explanatory theory for 
behavioral, descriptive decision making. We emphasize programs of research 
which develop integrative models of the psychological mechanisms which produce 
character f-stic behavioral phenomena in deductive and inductive inference, risk 
assessment and preference, and choice behavior in uncertain and incomplete 
information environments. 

D. Distributed Tactical Decision Making ; Objective is to develop an 
understanding of multiperson, cooperative decision making in complex, dynamic 
information environments. Research issues of current interest include 
strategies for decomposing complex problems leading to effective organiza- 
tional arrangements; strategies for aggregating judgments, plans, and 
estimates over multiple decision makers leading to consensus-oriented choice; 
an^ strategies for concurrency models in rapidly changing, multi-operator 
environments. 

E. Special Projects ; These are exploratory development projects which 
extend the basic research program toward Navy application areas. A 
Man-Machine Systems Technology program includes research in three topic areas; 
supervisory control, design for maintainability, and decision-making in command- 
control systems. A Small Business Innovation Research program includes 
research with personalized/prescriptive decision aids in support of complex 
information-processing and decision-making task performance. 
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A. MAN-MACHINE SYSTEM INTERACTION 



Research issues addressed in this cluster minimize concerns with 
the physical features of the man-machine interface and stress issues 
involved in cooperative, cognitive interaction with computer-based systems. 
Representative research topics currently include dialogue design character- 
istics, command and documentation language relative to software, resource 
theory for input/output device selection, perceptual theory implications 
for human-computer interaction in image processing, and cognitive consistency 
as a principle for interacting effectively with expert systems. 



2 



PILOT PERFORMANCE IN VERTICAL AND TRANSLATIONAL INSTRUMENT FLIGHT 
NR 196-170 

New Mexico State Unlversity/Roscoe 



Scientific Objectives ; T) e objective of this research is to develop a 
multiple regression model of helicopter and VTOL (vertical take-off and 
landing)pilot performance as a function of a l^rge number of critical 
real-world variables and display dynamics variables, for aircraft having 
variable control dyrairics. 

Approach : The experimental approach involves the identification of flight 
information requirements, display and control design variables, and critical 
maneuvers representative of various classes of low-altitude, low-speed 
missions typical of VTOL flight operations. Experiments are conducted in an 
aircraft simulator, a variable-characteristic device that allows simultaneous 
manipulation of display and control system configurations and dynamics. To 
facilitate the investigation of relatively large numbers of system design and 
task variables simultaneously, economical fractional-factorial screening 
experiments precede the application of central-composite response surface 
designs • The resulting generalizable display and control design principles 
will provide guidance applicable to aircraft capable of vertical as well as 
translational flight with a high degree of maneuvering independence in six 
degrees of freedom. 

Progress ; The flight simulator has been programmed to simulate a generic 
VTOL aircraft with dynamic performance characteristics selectable over wide 
ranges of values by the experimenter. An integrated vertical and horizontal 
display and control system with selectable graphic and dynamic 
characteristics was also simulated. The horizontal display, which presents 
ail information needed for precise vertical as well as translational 
maneuvering control, received the bulk of experimental attention and 
optimization early in the program with attention shifting to the forward- 
looking vertical display during 1985. Multiple regression models of the 
effects of experimental factors have been derived for each of three pilot 
performance measures. 

Potential Applications : Initial application of the results of this program 
has been the transfer of software for display and control system generation to 
the Naval Air Development Center for further evaluation in advanced 
helicopter and VTOL simulations. There is a possibility of a similar 
transfer at NASA-Ames Research Centor. 

Funding ; ONR; Naval Air Syste>n Command. 

Reports : 

Trujillo, E. J., & Roscoe, S. N. (1985) Horizontal display for vertical 
flight; A direction of motion experiment . (Tech. Report BEL-85-1/ONR-85-1) . 
Las Cruces, NM: New Mexico State University, Behavioial Engineering 
Laboratory. 
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Wiedemann, J., & Roscoe, S.N. (1985). A multiple-regression model of pilot 
performance in vertical and translational flight . (Tech. Report BEL-85-2/0NR- 
85-2). Las Cruces, NM: New Mexico State University, Behavioral Engineering 
Laboratory. (AJ}--A161364) 

SUPERVISORY CONTROL OF MAN-MACHINE SYSTEMS 
NR 196-179 

Massachusetts Institute of Technology/Sheridan 



Scientific Objectives ; This research investigates operator functions and 
performance in the supervisory monitoring and control of computer augmented 
man-machine systems. The technical objectives are to refine theories and 
construct models of operator performance that lead to a better understanding 
of factors affecting operator-computer interaction required for 
controlling dynamic system processes. 

Approach; The technical approach involves both model formulation and 
man-in-the-loop experimentation. The modeling efforts focus on achieving 
representations of the operator's understanding of system processes and the 
representation of rules and knowledge underlying system functions. 
Experiments are conducted to test the stability of these models and to 
investigate parameters derived from the models that (a) mediate communication 
(queries and command inputs) and (b) facilitate effective feedback on system 
status. Experiments involve laboratory simulation of operator tasks for 
control of vehicles and dynamic plant processes. 

Progress; Research during the past year has focused on the development of a 
structure and quantitative methods to represent component knowledge for 
computer models of dynamic systems and for mental models of the system 
operator. That work led to the concept of knowledge correspondence or 
calibration between the computer's model and the operator's model; 
measurement methods have been derived for assessing the degree of 
calibration. Experimental paradigms have been developed using action-state 
matrices which represent system operation in terms of input-output 
relationships. Experiments are being conducted; (a) to investigate methods 
and languages for the operator to communicate with the computer with regard 
to goals and understanding of system functions; (b) to assess factors 
that affect the calibration between the operator's and computer's models; and 
(c) to explore strategies for collaborative operator-computer interaction. 

Potential Applications; The theory and models being developed will have 
potential application for enhancing operator planning and monitoring 
performance in advanced Naval systems employing supervisory control concepts. 
Underwater search and inspection vehicles, remote manipulators, and process 
control systems (e.g., power generation and propulsion systems) are examples 
of classes of systems to which new theories of supervisory control may be 
applied with the potential of significantly improving operational perfomance. 
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Report ; 



Sheridan, T.B., Roseborough, J., Charney, L. & Mendel, M. (1985). 
Interaction of human cognitive models and computer-based models in 
supervisory control (Tech. Rep.). Cambridge, MA.: Massachusetts 
Institute of Technology, Department of Mechanical Engineering. 

USER-COMPUTER INTERFACE FOR HIGH RESOLUTION IM-VGE PROCESSING 
NR 196-182 

The Catholic University of America/Howard 

Scientific Ob j ectives ; In recent years many interactive Image processing 
algorithms have been developed for transforming, enhancing, and 
reconstructing imagery data, but little research has addressed the ability of 
human operators to use them effectively. This project investigates 
both the perceptual consequences of various enhancement operators and the 
strategic or planning activities of users in this user-computer environment. 

Approach ; User/computer performance is examined in localization, recognition, 
and other image processing tasks. Static imagery is used to investigate the 
perceptual aspects of enhancement operators vhereas dynamic, interactive tasks 
are used to investigate the strategic aspects; of interactive imaging. 
Standard psychophysical measures are used to assess performance; operator 
transition networks and production rules are used to characterize the 
user-computer interaction. 

Progress : Several experiments have been completed. One series has examined 
the role of spatial scale in the ability of observers to localize and 
classify simple gray-level images. Another has investigated how users apply 
enhancement operators in an interactive classification task. Additional 
follow-up experiments are underway. 

Potential Applications ; Interactive image processing systems are employed in 
anti-submarine warfare, anti-air warfare, surveillance, weather forecasting, 
and other Navy applications. The specification of perceptually-and 
cognitively-based user protocols for interactive Imaging is useful in the 
design of such systems. 

Archival Publications; 

Howard, J.H., Jr., Shaw, P.J., Robertson, S.P., & Raver, J. A. (1985). 
User strategies in interactive image processing (p. 31-32). In 
Proceedings of The Human Factors Society 29th Annual Meeting. Santa Monica, 
CA; The Human factors Society. 

Howard, J. H., Jr. (1985). Spatial scale in detection and recognition: 
Implications for imag^ processing (p. 1114-1117). In Pr oceedings of The 
Human Factors Society 29th Annual Meeting. Santa Monica, CA: The Human 
Factors Society. 
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THE STRUCTURE OF INFORMATION IN SOFTWARE SYSTEMS 
NR 196-191 

George Mason Universlty/Boehm-Davis 



Scientific Objectives; Relational and associatio.ial models of programmer 
performance are investigated for an understanding of the forms in which data 
bases associated with software development are stored in the memory of 
programmers. Predictions of programmer performance are made and tested on 
data bases organized around hierarchical, network, and relational structures. 

Approach; Software codes for three types of problems (resource al?ocation, 
sensor integration, and editing of address lists) are modified by experienced 
programmers according to a set of new specifications. The software code is 
presented in 3 different formats that vary along a representational dimension 
that depicts the decomposition of the problem; data-structure techniques, 
object-oriented techniques, and functional techniques. The effectiveness of 
each format type is experimentally evaluated in terms of the amount of time 
required to complete the modification, the number of errors made, and the 
number of modules that make up the software code which requires change. 

Progress: Data have been collected on software programs decomposed with 
the three methodologies into data structures, functional structures, and 
object-oriented modules. The effectiveness of those decompositions has been 
evaluated using complexity and effort metrics. The modif lability of those 
software programs was assessed. Mental models utilized by the programmers 
have been inferred, cognitive- interface principles have been defined, and the 
dependencies between problem type and the quality of the resultant software 
program have been determined. 

Potential Applications; Improvements in the understanding of the processes 
of programmer performance will lead to increases in software quality, 
productivity, and the design of techniques and tools that ase re that the 
foregoing goals are met effectively. 

Reports : 

Archival Publications : 

Boehm-Davis, D,A, (1985), Methodology and problem representation in 
programming. In Proceedings of The Human Factors Society 29th Annual 
Meeting , Santa Monica, CA; The Human Factors Society, 

Boehm-Davis, D,A,, & Ross, L,A, (1985), Program design methodologies: 
Structuring the software development process. In Proceedings of the IEEE 
Systems, Man, and Cybernetics Annual Meeting . New York; IEEE. 
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HUMAN FACTORS IN RULE-BASED SYSTEMS 
NR 197-078 

PAR Technology Corporation/Lehner 



Scientific Objectives : This effort investigated the psychology of user 
interactions with knowledge-based systems. The focus of the work was on 
cognitive aspects of user/expert system interactions. The specific objective 
was to identify cognitive variables that robustly result in high-quality user 
Interactions with knowledge-based systems. 

Approach : Several experiments were performed comparing combined user /expert 
system performance under a variety of cognitive conditions. The primary 
variables manipulated were (1) cognitive consistency between the problem 
solving procedures encoded in the expert system and those utilized by the 
user, and (2) the user's cognitive model of how the expert system operates. 
Three experiments examined the interrelationship between cognitive consistency 
and the cognitive model in a small expert system for selecting alternative 
business stocks. A fourth examined the impact of the cognitive model on a 
user's ability to operate a medical diagnosis expert system. 

Progress: The primary result of these four experiments was to show that a 
user's cognitive model has a dramatic effect on a user's ability to effectively 
interact with an expert system. Participants with a good cognitive model 
consistently and significantly outperformed participants with a poor cognitive 
model. This was a dramatic result since the experiments used a superficially 
weak manipulation of the cognitive model (viz., two pages of additional 
instruction that described general features of expert systems processing). 
The impact of cognitive consistency depended on the cognitive model variable. 
When participants had a poor cognitive model, consistency between the user's 
anl machine's problem-solving approach led to more effective user interactions 
with the system. When participants had a good cognitive model, consistency had 
either a negative or a negligible impact. 

Potential Applications : These results have significant implications for 
the design of the user interface of a knowledge-based system and other types 
of decision aid. First, if users are trained to have a good cognitive model 
of system processes, then those users will be able to adapt to a variety of 
alternative user interfaces. At the same time users with a poor cognitive 
model may not be able to effectively utilize the system, even if it has a 
so-called 'user friendly' interface. This suggests that the current concern 
with 'user friendly' interfaces is overemphasized. Alternatively, it suggests 
that a necessary ingredient of a 'user friendly' interface is that it promote 
a good cognitive model in users. 

Reports: 

Hall, R. (1985). Mental models and problem solving with a knowledge-based 
expert system (PAR Report No. 85-100). Reston, VA: PAR Government Systems 
Corporation. 

Lehner, P.E., Rook, F.W. & Adelman, L. (1984). Mental models and cooperative 
problem solving with expert systems . (PAR Report No. 84-116). Reston, VA: 
PAR Government Systems Corporation. 
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Lehner, P.E. and Zirk, D.A. a985) . Cognitive f.%:tors In user/expert system 
Interaction (PAR Report No. 85-107). Reston, VA: PAR Government Systems 
Corporation. 



Lelmer, P.E., Zirk, D.A., Kali, R.B. & Adelman, L. (1985). Human factors in 
rule-based systems; Final report (PAR Report 85-109). Reston, VA: PAR 
Government Systems Corporation. 



COMMITTEE ON BASIC RESEARCH IN HUMAN FACTORS 
NR 196-167 

NAS-National Research Council/Deutsch 

Scientific Objectives ; This effort seeks to provide new perspectives on 
theoretical and methodological issues in human factors and to identify 
research and other mechanisms to strengthen and expand the scientific basis 
of human factors research and development. 

Approach ; The Committee is composed of 14 members with professional 
specialties in human factors and allied disciplines of psychology, 
engineering, cognitive and computer sciences, physiology, and sociology. The 
Committee is augmented by Working Group and Workshop Timbers who provide 
intensive reviews of the state of the art and develop recommendations for 
advancing scientific theories and research applicable to human performance in 
human-machine systems. 

Progress ; Primary focus has been on the preparation of reports on a number 
of specific human factors topics for which there is a need for fundamental 
data and sound theoretical concepts. The reports address areas of human 
performance, such as methods for software design, human factors engineering 
data delivery systems, user-computer interaction, behavioral sciences in 
simulation, and supervisory control of complex systems. Also 
the Committee has established working groups and conducted workshops on the 
following topics; simulation technology to support human factors research 
development; design of computer software and effectiveness of programmer 
performance; the etiology of flight simulator sickness; and cognitive factors 
affecting the design of technical information systems. Other topics that are 
being examined include: theories and models ol human performance in 
man-machine systems; cooperative and distributed decision performance; 
visual performance with multicolored display systems; and integrated ergonomic 
modeling. 

Potential Applications ; The results of this work will strengthen the 
scientific basis underlying the enhancement of human performance in high 
technology environments and will serve to stimulate the scientific community 
to address new theories and models of human performance. 

Funding ; ONR, AFOSR, ARI, NASA, and NSF. 



Archival Publications: 



National Research Council (1984). Research and modeling of supervisory 
control behavior : Washington, DC: National Academy Press. 

National Research Council (1985). Methods for designing software to fit 
h uman needs and capabilities . Washington, DC: National Academy Press. 

National Research Council (1985). M uman factors aspect of simulation . 
Kashingtop, DC: National Academy ' ^ss. 



****** 
THE FOLLOWING WORK UNITS WERE COMPLETED DURING THE PAST YEAR: 



DESIGN PRINCIPLES FOR THE SPECIFICATIONS OF COMPUTER SOFTWARE PROGRAMS 
NR 196-175 

Performance Measurement Associates, Inc. , /Connelly 



Scientific Objectives : Interface principles were identified for software aids 
that contribute effectively to the joint efforts of system users and software 
analysts during the task of establishing requirement specifications. 
Theoretical issues investigated included: (a) Search techniques for 
non-programmer users; (b) Level of vocabulary appropriate for user-analyst 
communication; and (c) Transformations of a requirement statement that 
permitted a more complete specification. 

Approach : An experienced software analyst worked interactively, via a computer 
terminal, with a specification writer who was a non-programmer but 
knowledgeable in the design of inventory systems, to develop a 
software-requirement specification. Experiments were designed to investigate 
the effectiveness of various aid designs for the achievement of a least-cost 
requirements specification. Time and effort to produce a requirements 
specification were measured for vocabulary levels that ranged from 
non-abbreviated commands in a restricted-function low-level vocabulary to a 
powerful vocabulary requiring minimal keying input. Various transformations 
of requirements specifications v:ere preserted to the user to assess their 
impact on the completeness of the specifications. The generality of the 
findings of the software design principles were validated by the 
performance of non-programmer writers of software requirements specifications. 

Results and Conclusions : Sophisticated software aidc^ are necessary to help 
the non-programmer user in the efficient search of laast-cost specifications. 
When user-analyst communications were conducted over a period of 1-4 hours, the 
increased time required to recall the more powerful aids and command 
abbreviations was not compensated by either their brief input time or improved 
efficiency. Further, that effect appeared to be a function of command type 
and usage. 
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Potential Applications : Productivity ard quality can be enhanced in the 
design and organization of software requirements; software aids that jointly 
serve the system user and software analyst can assist in the achievement of 
those design goals. 

Technical Report ; 

Connelly, E.M. (1986, Jun<?). Investigation of the process of developing 
requireme nts specific ation s: Final Report. Reston, VA: Performanc e 
Measurement Associates, 



THE STRUCTURE OF INFORMATION IN SOFTWARE CODE AND DATA BASES 
NR 196-183 

Genera] Electric Company/Boehm-Davis 



Scientifi c Objectives : Design parameters or:.ented by data structure, the 
object, or the function of the problem, were examined for thei- effectiveness 
in the organization and integration of information. 

Approach; Working programs for non-trivial problems were produced from 
specifications by the use of alternate techniques that emphasized data 
structure, the object, or the function of the problem to be solved. The 
production code developed by a group of experienced programmers under 
the experimental conditions was evaluated for its quality' , the effort 
required to produce it, its use of machine resources, and the consistency 
of the problem solutions among the participants. 

Results and Conclusions; Objected-oriented methodologies led to better 
performance in the updating of software modules than techniques organized 
around either data structures or operations such as functional 
decomposition. Object-oriented techniques rely partially on data structures 
and partially on operations as the basis for structuring programs. The 
objected-oriented methodologies allowed programmers to adopt strategies for 
dividing the task of maintaining software code into more manageable parts 
consistent with their internal representation of that task. This practice 
permitted reductions in design time and in the complexity of the design 
solution, and the improvements were more effective with specific types of 
program tasks. 

Potential Applications : The identification and validation of principles for 
structuring a software program, speciticdtion of software-developed 
guidelines, and the determination of cost-effective trade-offs for that 
development, will assist significantly in the task of software acquisition 
and management. 

Reports; 

Boehm-Davis, D.A. (October 1984). Ap proaches to structuring the software 
developmen t process (Tech. Rep. GEC/LIS/TR~84-B1V-1) . Arlington, VA; General 
Electric Company, Data & Information Systems. (AD A147694) 
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Roehm-Davis , D,A. (October 1984). On the structure of information in software; 
Final Report (Tech. Rep. GEC/DIS/TR-8^-BlV-2) . Arlington, VA: General 
Electric Company, Data ft Information Systems. (AD A147580) 



Archival Publications ; 

Boehm-Davis, D.A. (1^84). On the structure of information in software (p. 
546). In Proceedings of The Human Factors Society 28th Annual Meeting s Santa 
Monies, OA; The Human Factors Society. 



THE IMPORTANCE OF SPECIALIZED COGNITIVE FUNCTIONING FOR TASK PERFORMANCE 
NR 196-186 

University of Pittsburgh/Gordon 

Scientific Objectives ; Determine whether a cognitive profile reflecting 
specialized tasks associated with the left or right cerebral hemisphere can 
be a useful predictor of job performance. Part I of the project was aimed 
at predicting successful Naval Aviators and other specialized jobs such as 
computer programmers. Part II focused on the relationship between the 
cognitive profile and job complexity of middle managers. 

Approach ; The cognitive profile was assessed with the Cognitive Laterality 
Battery (CLB) . The Battery is a group of tests selected to assess functions 
attributed to the right and left hemispheres of the brain. The CLB was 
presented on 35mm slic'es and audio tape to groups of participants. Performance 
on each subtest as well as on each of 2 factors, the difference between the 
factors, and the sum of the factors were the independent (predictor) variables. 

The dependent variables in Part I were the successes and failures of 
student pilots. These were obtained in 3 ways; 1) by success or failure in 
the program, 2) performance ratings obtained by flight instructors during 
training, and 3) flight scores from the Aviation Flight Jackets. In Part II 
the dependent variable was job complexity, which is defined as the number of 
different task components as well as the inter-dependence of one component on 
the next in a given job. About 1100 student naval aviators were tested for 
Part I; and about 120 individuals from business and industry were tested in 
Part II. 

Results and Conclusions ; Results demonstrated a significant superiority in 
visuospatial skills normally attributed to the right hemisphere for those who 
completed the aviation program. The same tests were given to individuals in 
management-related occupations; results did not clearly support the idea that 
successful managers were those with a visuospatial profile. 

Potential Appli c ations ; The application of specialized cognitive function 
tests will be ful for designated job-selection and job-match situations. 
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Reports ; 

Gordon, H.W., Curnow, CM., & Leighty, R. (1985). Specialized cognitive 
function as a risk factor in combat pilot selection , presented at the Annual 
Meeting of the International Neuropsychological Societ Denver, CO. 
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B. PERCEPTION 



Perception is viewed as computation and the projects in ch.'s cluster 
begin to develop rhis theoretical approach fo understanding early processes 
in human vision. The projects seek to combine computational modeling and 
psychophvsical experiments at successive stages of data representation that 
begin with retinal intensity arrays and progressively conL.truct the dynamic 
three-dimensional visual world. Computational theories provide both psy- 
chology and neurophysiology with testable models of the processes and imple- 
mentation mechanisms that underlie human vision. This program is intended to 
result in an understanding of human visual processing sufficient to guide 
manipulation of artificial visual environments, e.g., computer-generated 
iip-gery, in ways that create natural perceptual responses. 
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INFERENCES FROM IMAGES 
NR 196-188 

Massachusetts Institute of Technology /Richards & Ullman 



Scientific Objectives ; To develop a mathematical theory of how a visual 

llllTolZ 2D We'JnJ' an/sD^^tructure. 
fluctuations ii .^r '"'^"%^'^^^: ^^^'"Pl^- given only the intensity 

tiuctuations in a region of an image, assess whether or not the intensitv 
values could have arisen only from a surface consisting of wa'eMr^otJon. 

AB£roach: This problem is investigated in four stages: (1) computer 
L'vcho h' ^ --lysis. (3) algorithm d'evelopmen a^d (4) 

psychophysics. The first step, now completed, is to generate a realis ic 
surfa e of water or wood using computer graphics. Such generation requires 
a reflectance and surface model that makes explicit all the perceptuaUv 

u)'':ho"e' ^^s:^rf^^^"^• .^^^ ^^^^^ appearj;tt:;ii 

U; those of the reflectivity function, and (b) those that characterize the 
of aT s":i%e1l- -'-'.'^^ ^-'-s are hemispheric illumin^t on 

nL M / o'^"" function, with the surface shape being the sum of 

non-linear sinusoids. For wood, the key factors are the appearance of 
lllTnlT] correspond to growth rings, intersected by line 

elements (wood grain) parallel to the major axis of these ellipses! Given 
Mrh^r experiments test whether they uniquely de. ribe 

either wood or water, and demonstrate that these properties "can indeed 
that '° ''"^^^y' algorithms have been written 

aJoit the n'^r"' f ' ^^^^^"'^^ ^^^^i^- ^ binary assertion 

about the nature of the surface patch viewed (i.e. water, wood, etc ) the 

behavior of these algorithms will be compared with psych;physi al observations 

dfJi^. r^^K algorithm mimic humaJ behavior. t^I m^del' 

describes how observers see texture, such as water or wood. 

lnSSo5rI'''M ""'^"^"g of water and wood have been completed using 
hLp !f J S^P?^" system. A wide range of surface reflectance functions 
have also been explored to determine possible "false targets" - surfaces 
that might look like water but have different physical properties Image 
SevelopeS' '° ''''' characteristic of 'ooS has been ' 

Potential A p plications ; This research will eventually contribute a 
foundation of basic knowledge about how to manipulate artificial imagery Jn 
systems P""Pt-l experiences desired in training or operational 
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THE INTEGRATION OF SPATIAL INFORMATION FROM SHADING. TEXTURE GRADIENTS AND 
SURFACE CONTOURS 
NR 196-189 

University or Oregon/Stevens & Beck 



Scientific Objectives ; To understand the principal strategies by which 
spatial information from multiple , substantially independent, visual "cues" 
is integrated into coherent internal representations. The spatial 
infcrr.ation is of several different types (e.g., distance, local surface 
curvature), and the precision and accuracy of each varies according to the 
viewing conditions. 

Approach : There are two parallel efforts: computational modelling and visual 
psychophysics. Two central computational problems are: conversion of spatial 
inforration across type and integration of information of each type which 
diffeis in precision or accuracy. Reaction time and masked tachistoscopic 
presentation are used to explore the time course of spatial judgments, 
and to determine which types of information are explicitly available in human 
vision. Functional coupling between representations will then be examined by 
bringing into opposition spatial information contributed from different sources. 

Progress : Over the past year the apparent depth in monocular images in both 
perspective and orthographic projection has been examined by means of a 
computer-controlled stereoscopic probe. The first goal was to determine to 
what degree Images that are specifically constructed to contain information 
about local surface orientation (but no depth information per se) give rise to 
a measurable variation in apparent depth. For comparison, stimuli were also 
examined in which perspective (producing a texture gradient or linear 
perspective effect) gave clear information about distance variation across 
the surface. The results in terms of apparent depth, while less in the 
orthographic images than in the perspective, demonstrated clearly that the 
visual system can transform surface orientation to distance information. 
Studies are underway on (1) how distance information is two-dimensional ly 
extrapolated from locations where surface-orientation information is 
presented, (2) how depth from stereo disparity is made comparable to 
monocularly-derived depth, (3) the quantitative nature of monocularly-derived 
depth, and how relative depth is converted to absolute depth, and (4) 
computational models for surface interpolation in light of experimental 
findings regarding surface extrapolation. 

Potential Applications : An application area is flight simulation, where 
spatial information from motion, stereopsis, texture, shading, surface 
contours and sc forth are often of differing validity and even brought into 
opposition. Understanding of the type and magnitude of functional 
interaction among these sources of information wi]l have application to 
simulator design. 
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INFERRING 3-D SHAPE FROM IMAGE MOTION AND OCCLUDING CONTOURS 
NR 196-194 

Oniversity of California, Jrvine/Hof fman 



__ V-/ aeveiop a formal theory of the recovery of 

three-dimensional shape from image motion and occluding contours- (2) to 
test the theory for psychological relevance using psychophysical experimencs; 
U; to develop a computer implementation of the theory. 

AE£|oach: The recovery of three-dimensional structure from images is viewed 
as fundamentally a process of inference. The premises of the inference are 
the image data, e.g., two-dimensional image motion or the two-dimensional pro- 
jections of the folds of visible objects. The conclusion of the inference is 
a description of the three-dimensional shape of the viewed object. Perceptual 
inferences are in general not deductively valid, i.e.. the conclusion is not 
logically guaranteed by the premises. Therefore, real-world constraints are 
determined which grant the inferences a fair degree of inductive strength. 
Precise statements are sought of what image information is sufficient in 
conjunction with the constre.ints . to minimize false interpretations. These 
statements Hve given mathematical proof. Psychophysical predictions that 
arise from the formal theory of the perceptual Inference are tested 
experimentally. 

Progress: The constraints of fixed-axis rigid motion and fixed-axis rigid 
motion with constant angular velocity have been considered for the recovery 
V three-dimensional structure from motion. Three orthographic views of three 
points are sufficient to recover the correct three-dimensional structure using 
fixed-axis motion, and four views of two points were sufficient In the case of 
constant angular velocity. We are now testing subjects to see which of these 
constraints, if any, is being used by the human visual system. It appears 
that constant angular velocity is not required by the human visual system for 
the recovery of three-dimensional sLru^ture. 

A formal definition of an observer was abstracted from the specific visual 
inferences studied. This definition of an observer plays the same role for the 
study of perceptual inferences that the definition of a Turing machine plays for 
the study of computation. This formal definition of an observer is used to 
visul/i^fe?encis^"^"^""^ abstractly while concurrently studying the specific 

Potential Applications; This research will provide knowledge useful for the 
construction of automatic vision systems. It will also provide knowledge 
about the human perception of three-dimensional shape which is of value in 
the optimization of the man-machine interface. Finally, the definition of an 
observer will allow the study of perceptual inferences wiirhout distraction by 
details of specific inferences, and could be of value in advancing theories 
of physics which make an essential appeal to the notion of an observer, e.g. 
quantu-n theory. 
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INFORMATION PROCESGING IN PRIMATE VISUAL CORTEX 
NR 197-083 

California Institute of Technolcgy/Van Essen 

Scientific Objectives; Higher visual functions are carried out by a mosaic 
of cortical visual areas in the occipital, temporal, and parietal lobes. The 
objectives are to understand how information is processeH within individual 
areas and how it is distributed through a hierarchy of areas via several 
functionally distinct streams. 

Approach; Three experimental approaches are used for the analysis of 
visual cortex in monkeys and rats. The first involves the use of computer 
graphic displays for determining the responses of single neurons to texture 
patterns of precisely specified content. The second uses a new 
optical technique for monitoring neural activity simultaneously at many 
different sites across the surface of the brain. The third uses computer 
graphics to generate two-dimensional and three-dimensional representations 
of the cortical surface, for use in identifying visual areas and tracing their 
interconnections . 

Progress; Studies on the neural basis of texture discrimination, done in 
collaboration with Bela Julesz., have yielded promising preliminary results. 
Recordings in visual area V2 of the macaque monkey, have encountered cells 
whose responses appear to encode local texture differences of a type that are 
pre-attentively discriminable to human subjects. Multi-channel optical 
monitoring led to the establishment of a new experimental setup. These 
initial phases are now completed, and preliminary recording experiments have 
been succe->sf ully carried out. The computer anatomy project has resulted 
in the first computer-generated two-dimensional maps of highly convoluted 
regions of the cortical surface. 

Potential Applications: Thes studies will help to bridge the large gap 
between our understanding of human vision, based mainly on psychophysical 
experiments, and our much more detailed knowledge of visual processing in 
non-human primates. A more thorough understanding of the neural mechanisms 
underlying pattern recognition, motion analysis, and the control of directed 
visual attention will lead to better design of visual displays ir training 
and operational equipment as well as a more effective use of the visual 
modality in robotics and image-processing applications. 



A MULTI-LEVEL COMPUTATIONAL THEORY OF STEREOSCOPIC VISJAL IMAGE 

PROCESSING 

NR 197-084 

Naval Ocean Systems Center-Hawaii/Uttal 



Scientific Objectives ; The understanding of the rules and perceptual 
processes activated in the visual perception of stimulus forms in noise 
to be determined for two and three-dimensional displays. The aspects of 
stimulus which Influence perception, the development of a mathematical m 
and its refinement are to be pursued in that sequence. 
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Approach ; Dynamic visual signals are presented on the surface of a 
dot-plotting oscilloscope under the control of a hybrid digital-analoe 
computer Psychophysical experiments are carried out to determine the 
relative detectability . discriminability . recognizability . and re- 
constructability (in noisy visual contexts) of various forms with changing 
dimensions of space and time. The geometrical form of the target varies as 
well as the microstructure of ar-angement of dots that compose it. The maior 
JrnJpH"''p complete automaticity of the entire experimental 

procedure from the generation of stimuli to the production of graphic displays 
of the data and the interplay between the psychophysical results and the 
computational model. 

Progress: A large nvmber of the geometric parameters of the stimulus forms 
are varied to determine the effect of shape on detection, discrimination, 
recognition, and reconstruction. Among many other results, three novel and 
general rules of display perception are emerging: (1) Straight lines are the 
most powerful stimuli in complex displays and are easily detected- 
(2; Humans detect surfaces best when the surfaces are sampled with 
randomly-placed dots rather than with regularly-placed dots; and (3) 
Three-dimensional shape is not a strong determinant of detectability even 
though two-dimensional shape is. All three of these results are novel and 
unexpected; the first two are predicted by the correlational model that 
guides the research program. The third rule is not anticipated by the model 
and points the way to a new generation of computational models for these 

^hese rules vary as a function of task (2D versus 3D) and 

sampling strategy. 

Potential Applications : These perceptual processes are useful in understanding 
Jhr.. .7 . the design of radar and 3onar visual displays in both two and 
three dimensions; these perceptual processes are also salient in radiology, 
gsology. atmospheric physics, and oceanography, where information must be 
extracted from noisy, grainy, and incomplete visual displays. 

Reports 

Uttal, W.R. (June, 1985). Test of a multi-level computation al theory of 
stereoscopic visual image processing (Rep. PEPILAB-6). Ann Arbor, MI- 
University of Michigan, Institute for Social Research. (AD A157011) 

Kincaid, W.M., & Uttal, W.R. (1986). The effect of 3-D orientation and 
39r392!396? °^ dotted planes. Perception & Psychophysics . 

COMPUTATIONAL ANALYSIS OF THREE-DIMENSIONAL FORM 
NR 197-088 

Brandeis University /Todd 



Scientific Obiectives: The objective is to develop a data base of 
psychophysical functions between stimulus variables and perceptual judgments 
as foundations for the computational analysis of human vision. The research 
is directed at understanding how human observers Infer 3-dimensional 
structure of objects in space from the patterns of light intensity witl.in a 
visual image. 
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Approach ; The investigi^tor is developing a library of systematically 
controlled computer generated images with texture, shape, ri2«=». shading, 
reflectance and illumination. Psychophysical experiments are conducted that 
relate stimulus variations to judgments of orientation, depth and Gaussian 
curvature. 

Progress ; Five separate experiments, which investigated the effects of 
optical texture and perspective, have been completed. The data from these 
experiments were evaluated in terms of various computational models that have 
been proDosed in the literature. A close fit of the data was obtained by 
adapting a neural network model of 2-dimens'.onal perceptual organization 
recently proposed by Grossberg ana Mingola. An extension of that model to 
3-dimenslonal form perception suggested a number of quantitative predictions 
which were confirmed in subsequent experiments. Work has started on the 
perceptual analysis of image shading. One important finding that nas emerged 
from pilot investigations is that abrupt changes in image intensity that 
occur at self-occluding boundaries are of fundamental importance for correctly 
interpreting smooth variations of intensity in neighboring regions. A new 
experiment is in progess to provide an empirical demonstration of this 
phenomenon. Observers' judgments of local surface orientation are obtained 
for identical regions of smoothly varying image shading in different contexts, 
both with and without self-occluding contours. 

P otential Applications : Two aspects of this research make it particularly 
relevant to a number of practical applications. Fir^t, the relative 
perceptual salience of a wide variety of image properties will provide useful 
human factors guidelines for the generation of computer graphics disp]ays. 
Second, comparing the abilities and limitations of actual human observers 
with those of existing computational analyses will suggest new directions for 
the analysis of visual information that may be useful to researchers in the 
field of machine vision. 

Funding ; NSF, ONR. 



STUDY OF HUMAN VISION USING CELLULAR AUTOMATA 
NR 197-089 

University of California, Berkeley/Glaser 



Scientif rc Objectives: To develop a computational model of binocular stereo 
depth perception and of motion perception using e formalism of cellular 
automata and parallel distrunted processing networks. These techniques are 
readily simulatab]e on standard computers and have the possibility of working 
in real time on future parallel computers. They seem particularly well 
suited to capturing the main features of the neural architecture of the 
piimate visual system and also to modeling at least some aspects of its 
performance. 

Approach; Psychophysical results from the literature wi]l be supplemented 
with new psychophysical experiments designed to help choose among possible 
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visual mechanisms to be simulated. This work will use a variety of 
computer-driven CRT displays and occasional vision experiments not using 
CRT s. Particular attention will be given to distinguishing among local and 
global effects for which the psychophyslcs. the neural mechanisms, and the 
computational models may be quite different but related in some way. 
These models will use hierarchies of two-dimensional cellular automata 
or other parallel distributed-processing networks specialized in executing 
certain functions at each level with rich forward and retrograde 
communication among levels. 

Progress; The investigators have shown that apparent depth of simple figures 
in the center of a field of view is strongly affected by simple stimuli quite 
far from the central region. Such global stereo effects have not been 
reported before. Vertical motion is strongly preferred over horizontal motion 
In certain ambiguous apparent-motion configurations. The first cellular 
automata models were not useful because they were not sufficiently constrained- 
almost any observed psychophysical result could be described well in many 
different ways. New constraints from anatomy and computational theorv are 
being examined. 

Potential Applications; Any computational model that successfully simulates 
a human psychophysical ability can serve a robot to do the same task. If the 
model uses cellular automata or other parallel distributed-processing 
techniques, there is a good possibility that a parallel computer can be built 
to perform at speeds comparable to or faster than human performance. Robots 
able to exhibit human performance at human or faster speeds will have many 
applications. 



COMPUTATION OF STEREO AND VISUAL MOTION FROM BIOPHYSICS TO PSYCHOPHYSICS 
NR SRO-202 

Massachusetts Institute of Technology /Poggio, Hildreth & Koch 

Scientific Objectives; To unc'erstand how the human visual system makes 
three-dimensional Inferences from dynamic, two-dimensional retinal images. 
Specific areas of interest are the computations underlying the analysis of 
visual motion and binocular stereo. 

Approach; The problems of binocular stereo vision and visual motion, and 
their fusion, are investigated through an interdisciplinary combination of 
computational theory and modelling, psychophysical experimentation, and 
neurophysiologicf.l analysis and experiments. This approach includes emphasis 
on; (1) the primitives for motion measurement and stereo matching; (2) the 
additional constraints needed to solve problems of measuring motion and stereo 
matching; (3) the detection of object bound^'ries from relative motion and depth; 
(4) recovery of 3D structure from motion and stereo and; (i,) fusing stereo and 
motion information. 

Recent theoretical developments in this research program suggest a common 
framework for most early vision problems and for the fusion of information 
from multiple visual sources. 
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Progress; Extensions of regularization theory have been exploited to 
develop algorithms for edge detection, spatiotemporal approximation, stereo 
and color computation. New structure^-from-motion schemes have been analyzed 
and compared with psychophysical experiments. Hybrid networks of analog and 
digital components have been devised for efficient implementation of 
stochastic rep:ularization algorithms for early vision problems. The 
biophysical basis of the computation of motion has been studied and 
experimental prediction has been perfected, together with powerful computer 
simulation tools. 

Potential Applications; This research contributes to the understanding of 
(and possible improvement on) human performance In visual tasks. Interesting 
applications can be developed for processing visual images based on motion 
and stereo information, integrating results of different visual processes 
performed naturally or artificially, integrating information from different 
sensors, and achieving fast — possibly real-time — image understanding in 
machines. 

Archival Report ; 

Poggio, T., Torre, V. & Koch, C. (1985). Computational vision and 
regularization theory. Nature, 317 , 314-319. 



IMAGE REPRESENTATION 
NR SRO-203 

New York University/Hummel & Landy 

Scientific Qpjectives; The research seeks to enhance tlie understanding of 
human vision by studying data structures for image representation. The 
concentration is on variations of the Laplacian pyramid image-representation 
scheme, which decomposes image data into multiple scales of resolution. 
Methods for labeling information in image representations are also 
investigated in this study. The goal is to develop inference methods that 
can combine information from multiple data sources. Ultimately, this research 
seeks to aid computer vision systems in object recognition and scene analysis 
In the presence of ambiguity and noise. 

Approach; By writing computer models of representation schemes, digital 
image data are transformed according to models of early visual 
representation. Three methods are used to study the validity of the 
representation. First, a mathematical analysis is made to determine tbe kind 
of information that is retained in the data structure, and how that 
information can be used to reconstruct an approximation to the original 
image. Second, by numerically computiiig such reconstructions, human observers 
In psychophysical experiments visually compare original data with 
reconstructed data, to assess the representation's ability to capture salient 
information in the image. Finally, methods for extracting features and 
inferring structure using the representation are studied. A successful 
representation makes explicit those features necessary for aggregating 
knowledge to interpret the image. 
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Attention hns focused on Lnplaclan pyrnmid data-structure variants 
and on iterative knowUdpe aggregation methods lor inferring information from 
various sources. Progress has been made on multi-resolution decomposition of 
image data based op orientatlon-speri f ic featuies. Nonlinear decomposiLiun 
methods have been imp! ericnted . Among Iterative inference methods, the Dempster/ 
Shafer theory of evidence has supplied new ideas for representing degrees of 
uncertainty, resulting in novel combination formulas. A new iterative 
multi-resolution model of stereopsir, computation has also been developed, 
incorporating those ideas on "spread of uncertainty." 

Potential Applications; Knowledge of the huiran visual representat^'on 
promises to aid image understanding tasks when building robotic systems 
capable of ,.achine vision. More generally, methods for inferring structure 
and combining information should find applications in expert systems and 
intellig2nt computer devices. 

Reports: 

Huuimel. R.A. (September 1985). Connected component labeling in i mage 
processing with MIMD architecture s (Robotics RPSParrh R^p /.q;> m.,, yr^,,. 
New York University, Courant Institute of Mathematical Sciences. 

Hummel, R.A. & Landy, M.S. (November 1985) A viewpoint on the t heory of 
evidence. (Robotics Research Rep. 57) New York: New York University. — 
Courant Institute of Mathematical Sciences. 

Landy. M.S. & Hunmiel, R.A. (September 1985) A brief survey of kn owledge 
aggregation methods. Rep. 51; New York: New York University. Courant Institute 
of Mathematical Sciences. 



Archival Publications! 



Carrlhill. B., & Hummel, R. A. (1985). Experiments with the intensity ratio 
depth sensors. Computer Vision, Graphics, and Image Processing . 32, 337-358. 



****** 
THE FOLLOWING WORK UNITS WERE COMPL'-TED DURING THE PAST YEAR: 



DISCRIMINABILITY OF SIGNALS FROM 
NR 197-070 

University of Michlgan/Uttal 



NOISE IN A DYNAMIC STFREOSCOPIC SPACE 



Scientific Objectives; An analysis of visual detection sought an 
understanding of the relative importance of the global geometry of forms when 
the local features of the fcnns were carefully controlled. The 
autocorrelation model that adequately predicts psychophysical performance 



22 

30 



with 2-dimensioTial dotted patterns was utilized as a first approximation in 
the development of a figure of merit for the detection of 3-D dotted patterns 
and to predict the relative detectabili ty of those forms. 

Approach : The detection of forms composed of a constellation of dots within 
a background o*" random dots permitted the examination of subtle 
information-processing attributes of visual processes. Two dimensional 
targets (single dot, line, or plane) were embedded in random visual dots 
arrayed in 3-D space and experiments measured their '^^uectability as a 
function of numerosity, position, repetitiveness , spacing, and orientation. 
The detection of single dotted lines was examined when they were masked by 
other dotted lines. The detection of various 3-D nonplanar forms composed of 
dots embedded in random visual dots was studied as a function of the spacing 
and density of the random dots, and the degree of deformation of the forms. 

Results and Conclusions : Specific rules were defined for :he perception of 
forms: (a) linear productivity - straight lines consisting of dots spaced 
equally from each other are detected better than when the dots are curved; 
(b) random sampling - response surfaces are detected better when they are 
constructed from random arrays of dots than from regular arrays when the 
regions n'^ir each other are more alike than their more distant regions; 
and (c) three-dimensional noncomputability - nonplanar surfaces, and 
presumably solid forms, cannot be processed by the visual system in the same 
way as two-dimensional objects; the nervous system does not seem to have the 
power to compute three-dimensional geometry. 

Potential Applications : The use of dotted forms is a useful paradigm for the 
evaluation of designs of radar and sonar displays in both two and 
three-dimensional representations. The experimental results are suggestive 
for the design of informational displays in radiology, geology, atmospheric 
physics, and oceanography, where usefu^ data must be extracted from noisy, 
grainy, and incomplete displays. 

Report: 

Uttal, W.R. (November, 1984). The discriminability of signals from noise in 
a dynamic stereoscopic space: Final Report (Tech. Rep. PERLAB-5) . Ann Arbor, 
MI: University of Michigan, Institute for Social Research. (AD A148406) 
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C. DECISION MAKING 



In the past ten years, repparch in human decision making has been 
trending away from comparisons of behavior with optimum economic or stat- 
istical models toward more descriptive, beaavioral characterizations. 
Numerous robust behavioral phenomena have been identified which are more than 
simple, sub-optimal approximations of optimal models, but appear as con- 
tradictions. i..consistencies. fallacies and biases. These phenomena require 
explanation by psychological theory and the emphasis of research in this 
cluster is to implement this shift from behavior comparisons and description 
toward explanatory theory. Currently supported research includes the 
development of psychological theory in diagnostic inference processes, risk 
assessment and preference, selection of decision-choice heuristics, and 
reasoning processes in risky choice. The projects emphasize algebraic or 
production-system models of psychological processes that have the potential 
to account for behavioral phenomena. 
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EFFORT AND ACCURACY IN DECISIONS 
NR 197-063 

Duke University/Payne 



Scientific Objectives ; This research is directed at a better understanding 
of the psychological mechanisms that lead to the task-contingent use of 
decision strategies. The major hypothesis Is that strategy selection Is 
a function of a strategy's ability to produce an effective response and 
demand for mental resources or effort. 

Approach ; A number of the heuristic decision strategies are decomposed into 
a set of common elementary Information processes (eips), such as comparing 
the magnitudes of two quantities. The eips are similar to those found in a 
variety of cognitive activities. Production system models of the heuristics 
are constructed. Monte-Carlo simulation is used to examine the effort and 
accuracy implications of the modeled heuristi'^s in a variety of choice 
environments. Empirical validation of the models is conducted using the 
process-tracing techniques of verbal protocols, monitoring of information 
acquisition, and response-time measurement. 

Progress ; Initial efforts were directed at showing the impact of context and 
task variables on decision behavior. More recently, an extensive Monte-Carlo 
simulation of choice heuristics for both risky and nonrisky decision tasks 
was completed. The simulation showed that certain decision strategies 
approximated trie accuracy of optimal procedures with substantial savings in 
effort. Effort and accuracy, however, were highly contingent upon 
characteristics of the task environment, such as number of alternatives and 
the presence or absence of dominated alternatives. An empirical study 
designed to test the validity of the decomposition approach to modeling 
decision heuristics, and to provide estimates of the effort associated with 
different eips, was completed. The study employed response-time measurement 
and a new computer-controlled methodology for monitoring information acquisition 
behavior. The results indicated that the total time to make a decision u^ing 
a specified strategy was highly predictable from counts of the eips. 
Additional experiments to validate the model of the effort and the accuracy 
of choice heuristics are underway. 

Potential Applications ; The Importance of task variables in the design of 
decision support systems is clear. The decomposition of strategies into 
sets of component elementary information processes, the computer simulation 
of production systems models, and process tracing experiments, will 
contribute to the design of more effective decision aids. 



Archival Publications : 

Johnson, E.J. & Payne, J.W. (1985). Effort and accuracy in choice. Management 
Science, 31, 395-414. 
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Laughhunn, D. J. , & Payne, J.W. (I98A). The impact of sunk outcomes on risky 
choice behavior. INFOR (Canadian Journal of Operational Research and Information 
Processing) , 22, 151-181. 

Payne, J.W. (1985). In G. Wright (Ed.), Psychology of risky decisions in 
behavioral decision making . New York: Plenum Press. 

Payne, J.W. , Laughhunn, D.J., & Crum, R. (1984). Multiattribute risky choice 
behavior: The editing of complex prospects. Management Science , 30, 1350-1361. 



COGNITIVE CONTINUUM THEORY OF HUIiAN JUDGMENT 
NR 197-073 

University of Colorado/Hammond 

Scientific Objectives ; The principal research objective is to compare 
directly the relative efficacy of three modes of cognition: intuition, 
quasi-rationality, and analysis, as they are actually employed in different 
task conditions. 

Approach : Twenty-one expert highway engineers were studied over three 
different tasks, each displayed in three different ways. Indices were 
developed for (a) measuring the location of each task condition on a 
continuum ranging from intuition-inducing to analysis- inducing, and for (b) 
measuring the location of each expert's cognition on a continuum ranging from 
intuition to analysis. A new experimental methodology, namely a 
combination of Campbell and Fiske's multitrait, multimethod matrix and 
Brunswik's representative design, was employed in order to evaluate the 
convergent and discriminant validity of each judgment under each condition. 

Progress: Individual analyses of each expert's performance 
over the nine conditions show that task-induced cognition is located at 
approximately the same location on the cognitive continuum Index for 
virtually every engineer. In contrast to the results found with the 
customary indirect comparisons, intuitive and quasi-rational cognition 
frequently outperformed analytical cognition in terms of empirical accuracy. 
Analytical cognition produced infrequent but large errors. Judgmental 
accuracy was related in part to the degree of correspondence betx^een the task 
location and the location of the expert's cognitive activity on the cognitive 
continuum. The methodology introduced in this study provides a new procedure 
for evaluating expert judgment. 

Potential Applications : The task continuum index can be used to measure the 
extent to which judgment of decision tasks induce intuition or analysis, thus 
guiding the choice of cognitive support systems. The procedure developed for 
measuring convergent and discriminant validity of judgments can be generally 
used for evaluation of, and location of faults in, expert judgment. 

Report: 

Hamm, R.M. (1985). Moment by moment variation in the cognitive activity of 
experts (Tech. Rep. No. 257). Boulder CO: University of Colorado, Center 
for Research on Judgment and Policy. (AD A159466) 
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STUDIES TN RISK: PERCEPTION AND PREFERENCE 
NR 197-079 

University of Wisconsin/Lopes 



Scientific Objectives : The theory to be tested proposes that perception 
of risk is related to the cumulative properties of risky distributions, 
with judgments of riskiness reflecting enhanced weighting of the worst 
outcomes in the distribution. Risk aversion is proposed to reflect a 
trade-off between avoiding risk and achieving some aspiration level. 

Approach : Six experiments concerning participant preferences for risky 
distributions are related to the theoretical model and to self-reports of 
short-term and long-term goals and strategies. The planned experiments are: 
(1) a study of aspiration level effects among risk-seeking and risk-averse (RA) 
participants; (2) a comparison of distributional preferences for potential 
gains and losses; (3) a comparison of risk preferences for monetary and 
non-monetary situations; (4) a study o[ risk attitudes under offensive and 
defensive sets in a competitive task; (5) an exploratory study of process- 
tracing techniques in risk processing; and (6) an exploratory study of a 
mathematical index for quantifying riskiness. 

Progress : The main results of Experiment 1 were (1) that reflection occurs 
reliably for both risk-averse and risk-seeking participants only when one of 
the gambles in the choice pair involves a riskless component (or, for RA 
participants, a sure thing), and (2) patterns of preferences for risk-averse 
and risk-seeking participants are not suggestive of pure risk styles, although 
the patterns are largely antithetical to one another. Instead, the data 
suggest that the two groups differ in their goals and their strategies for 
satisfying competing needs. Analysis of protocol data from Experiment 5 
suggested that risk-aversion and risk-seeking attitudes reflect primarily (1) 
aspiration-level effects for all participants and (2) security seeking for 
risk-averse participants versus potential-see.cing for risk-seeking 
participants. The primary result of Experiment 3 was that participant 
preferences for distributions are quite similar for monetary and non-monetary 
scenarios, although differences ■'n goals produced some shifts of preference. 
Data collection in progress for Experiment 4 suggests that, in competitive 
tasks, preferences among moves differing in the distribution of possible 
outcomes will be determined primarily by current needs (i.e., board position) 
rather than by simple aversion or attraction to risk. Theoretical work is 
progressing on a model of risk embodying the effects of aspiration level and 
security/potent ial motivation. The model is Intended to include an index of 
dictributional security/potential (Experiment 6). 

Potential Applications : General understanding of risk perception and 
processing may contribute to being better able to predict and understand 
behavior under risk in real situations. Psychological assessment of risk 
attitude may contribute to the selection of personnel for risky operations 
and for training of personnel who must operate under risk. 

Report ; 

Schneider, L. & Lopes, L. L. (1985). Reflection in preferences for 
multioutcome lotteries (Tech. Rep. WHIPP 22). Madison, WI: University 
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PROBABLE CAUSE: A DLCISION MAKING FRAMEWORK 
NR 197-080 

University of Chicago/Einhorn & Hogarth 

Scientific Objective s: Causal inference is an essential cognitive activity 
that combines judgments of causation with judgments under uncertainty. Thus, 
causal inference Involves judgments of probable cause. Since uncertainty and 
causality are intertwined in most real-world situations, the goal is to 
understand how, and how well, people combine these two modes of thought. 

Approach: A model is developed for understanding how judgments of probable 
cause are formed. The model consists of three major elements: (1) the 
causal strength of a variable is first shown to depend on its being a 
deviation or difference in a background or field (much like a f igurt/ground 
relation); (2) various cues-to-causality , such as covariation, temporal 
order, contiguity in time and space, and similarity of cause and effect, are 
discussed as probabilistic indicators of causal relation. A model for 
combining the cues and background is developed that captures many aspects of 
causal inference; and (3) the discounting of the strength of an explanation on 
the basis of specific alternatives is considered as a special case of the 
sequential updating of beliefs. 

Progress: The model for evaluating the strength of a causal explanation 
accounts for many interesting and heretofore diverse aspects of causal and 
statistical inference. For example, the concepts of "spurious correlation", 
"illusory correlation", confusion of inverse probabilities, confusion of 
joint and conditional probabilities, violations of the conjunction rule for 
the probability of two events, and so on, are brought under one conceptual 
approach. Furthermore, the model can be extended to more complex issues 
having to do with scenario coherence, types of causal chains, and the use of 
imagination in constructing causal explanations. 

Potential Applications : This work can be applied to any decision-making 
context where incomplete and uncertain evidence must be evaluated with 
respect to causal implications. 

Reports ; 

Einhorn, H. J. (1985). A model of the conjunction fallacy (Tech. Report). 
Chicago, XL: The University of Chicago. 

Einhorn, H. J., & Hogarth, R. M. (1985). A contrast/surprise mod el for 
updating beliefs (Tech. Report). Chicago, XL: The University ci Chicago. 

Hogarth, R. M. (1985). Why bother with experiments? (Tech. Report). 
Chicago, XL: The University of Chicago 

Hogarth, R. M. & Kunreuther, H. (1985). Risk, ambiguity, and insurance 
(Tech. Report). Chicago, XL: The University of Chicago 
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Archival Publications ; 

Einhorn, H. J., & Hogarth, R. M. (1985). Ambiguity and uncertainty in 
probabilistic inference. Psychological Review , 92^, A33-A61. 

Einhorn, H. J. , & Hogarth, R. M. (1986). JvHging probable cause. 
Psychological Bulletin , £9, 3-19. 

Hogarth, R. M. & Kunreuther, H. (1985). Ambiguity and insurance decisions. 
American Economic Review, (AEA Papers and Proceedings), 75, 386-390. 



DECISION AND JUDGMENT 
NR 197-081 

Stanford University /Tversky & Kahneman 

Scientific Objectives : The development of a descriptive analysis of judgment 
previous work on judgment under uncertainty and prospect theory. 

Approach : The traditional approach to the study of decision making has been 
normative in character and structural in form. That is, it derives from 
rational or logical analysis of decision and judgment, and it characterizes 
the structure of preferences and beliefs. In contrast, here a descriptive 
approach is adopted that is constructive in nature. This approach attempts to 
account for the manner in which people construct judgments and decisions. 
Because the act of constructing preferences and beliefs depends on the manner 
in which the problem is framed, the invariance condition that characterizes 
structural theories does not always hold. 

Prog-^ess : (1) Analysis of transparent and opaque decision problems: A 
systemar.ic empirical analysis of risky choice reveals that human decisions 
violate not only the strong conditions of cancellation (or independence) and 
transitivity; they also violate the most basic conditions of (stochastic) 
dominance and invariance. Because the latter are essential to any so-called 
rational model, the discrepancy between the normative and the descriptive 
analyses cannot be reconciled. On the other hand, the same axioms are 
generally satisfied in transparent situations; i.e., problems that are 
framed so that the applicability of the axioms is clear. (2) Multiattribute 
prospect theory; An analysis of different choice structures has been 
developed: (a) positive choices between two goods, or between a monetary 
gain and a good; (b) the "goods-money exchange" in which a good is given up 
in exchange for money; (c) negative choice between two losses or otherwise 
aversive consequences; (d) the "money-goods exchange" in which money is 
given up in exchange for a good. The comparison of (b) and (d) provides 
a sensitive measure of loss aversion (the typical manifestation of loss 
aversion is a general reluctance to trade). The comparison of structures 
(a) and (c) provides a measure of relative loss aversion. We have confirmed 
loss aversion and assessed relative loss aversion for a number of different 
attributes. The pattern of results is robust, indicating very extreme loss 
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aversion for increases in risks (e.g., safety or health), and rather Mttle 
loss aversion for leisure (e.g., vacation days) relative to money. (3) Designs 
for probability ^udgment: theories of subjective probability are viewed as 
formal languages for analyzing evidence and expressing degree of belief. 
Two probability languages, the Bayeslan language and the language of belief 
functions, were analyzed from that perspective. A comparison of the semantics 
(i.e., the meaning of the scale) and the syntax (i.e., the formal calculus) 
of these languages suggests different approaches to probability judgment. 
The concept of a design for judgment of probability, analogous to the notion 
of the design of an experiment, is developed and applied to a few evidential 
problems. 

Potential Applications; The role of framing, the distinction between 
transparent and opaque problems, the analysis of loss aversion, and the 
concept of design for probability judgments all provide descriptive and 
analytic tools for evaluating and debugging human judgment and decison making. 

Funding : ONR, AFOSR. 
Archival Publications ; 

Shafer, G. & Tversky, A. (1985). Languages and designs for probability 
judgment. Cognitive Science , 9, 309-339. 

Tversky, A. & Kahneman, D. (1984). The fr?ming of decisions. Covello 
et al. (Eds.), ASI Serie s, Vol. G4, NATO. 



****** 
THE FOLLOWING WORK UNITS WERE COMPLETED DURING THE PAST YEAR: 



TASK INFLUENCES IN THE ANALYTIC- INTUITIVE APPROACH TO DECISION MAKING 
NR 197-074 

Rice University/Howell 



Scientific Objectives; Experiments sought to determine the effects of task 
variables on human judgment and decision behavior. Variables were chosen and 
consequences predicted largely on the basis of Hammond's cognitive continuum 
theory, using tasks designed to be neither exclusively "intuit ion- inducing" 
nor "analysis-inducing". 

Approach; Experiments are conducted in a variety of problem contexts. The 
emergency resource allocation problem requirea acquisition and use of' 
observed event-frequency patterns. The optional stopping problem involved a 
succession of choices between drawing more information and taking terminal 
action. The personnel decision and threat evaluation problems were multiple 
cue-weighting tasks of the sort used In "policy-capturing" research. The 
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two-stage judgment task permitted the study of cue-weighting strategies as a 
function of various levels (and kinds) of input pre-processing. 



Results and Conclusions : Findings were fairly consistent with Hammond's 
theory, although the research was not intended as a direct test of that model. 
More importantly, a number of task features were shown to influence judgment/ 
decision performance reliably — in some cases, across task scenarios; in others, 
in a selective fashion. For example: judgment was affected in both qualitative 
and quantitative ways by the manner in which information was displayed. Graphic 
coding promoted more "holistic" processing than did numeric coding, and 
consequently, better decisions under "pressure" situations. Time pressure, 
amount of information, rule availability, and rule complexity did not produce 
the "across the board" effects suggested by continuum theory. Each affected 
some aspect of performance on some task scenario, but not in the coherent 
fashion one would expect if an individual's cognitive approach were merely 
shifted along a continuum. 

Potential Applications ; Care should be taken in choosing the features of 
decision systems (such as display formatting, task sequencing, highlighting, 
feedback mode), since these can and do affect many aspects of the decision 
process. Graphic coding is particularly useful where conditions dictate 
"intuitive" judgment. Manual preprocessing is a promising alternative to 
aiding in some decision settings. 

Reports ; 

Fontenelle, G. & Howell, W. C. (1985). A replication and extension of 

the inducement of the availability heuristic (Tech. Rep. No. 8^-5). Houston, 

TX; Rice University, Dept. of Psychology, December, 1985. 

Friedman, L. , Howell, W.C. , & Jensen, C.R. (1984). Diagnostic judgment 

as a function of the preprocessing of evidence (Tech. Rep. No, 84-4). Houston, TX 

Rice University, Dept. of Psychology, October, 1984. 

Howell, W. C. (1985). Task influences in the analytic intuitive 

approach tc decision making (Final Repoit). Houston. TX: Rice University, 

Dept. of Psychology, December, 1984. 
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D. DISTRIBUTED TACTICAL DECISION MAKING 



This is a Special Focus Program sponsored Jointly by ONF and the Space 
and Naval Warfare Systems Command (SPAWARS). It is a multidisciplinary 
research effort involving psychological, mathematical, computer science and 
communicat-'.ons approaches to basic issues in multi-person, cooperative decisi -n 
making. The program is intended to provide scientific underpinnings in the 

u7u^^ 1^^°""^/^: "^'^"'^^ ^"'^ functional relationships among key variables 

which define decision making in geographically dispersed, decentralized command/ 
control systems. Initial research topics of concern include strategies for 
decomposing complex problems that lead to effective organizational architectures, 
strategies for aggregating multi-person judgments, estimates and plans leading 
to effective consensus, and strategies for maintaining data-base concurrency in 
dynamic, multi-operator environments. The program emphasizes measures of 
effectiveness for distrib'ut<>d decision systems with special attention to the 
issues of reconfiguration and reconstitution. 
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RESEARCH IN DISTRIBUTED DECISION MAKING WITH APPLICATIONS TO C SYSTEMS DESIGN 
NR 649-001 

Alphatech, Inc. & University of Connecticut/Castanon, Entin, Serfaty, & Kleinman 

Scientific Objectives ; The objective is to develop contributions to an 
integrated theory of distributed tactical decision making based op 
theoretical and experimental research in mathematics and psychology. 

Approach ; The work will consist of two major tasks. The first task will 
develop mathematical models in team and game theory with emphasis on 
developing quantitative methodologies for analyzing problems of dynamic 
Decision making under uncertainty. Under this task, team decision problems 
wi]l be investigated as distributed optimal stopping problems and dynamic 
stochastic games. Conditions characterizing optimal team strategies and 
performance in these decision problems will be investigated. The second 
task will focus on the development of normative/descriptive uiodels of human 
decision making. This tank will build on theories and models of behavioral 
decision making. 

Progress ; The mathematics component of the research program has focused on 
decision algorithms for f inite^-state, discrete-time, partially observable Markov 
decision processes, characterized by uncertainty in the state due to noisy 
measurements. The basic algorithm is stochastic dynamic programming which; 
(a) involves the temporal decomposition of a dynamic problem into a sequence 
of static problems; and (b) separates estimation and control functions. The 
Smallwood-Sandik formulation is being used in the development of decentralized 
control models. The human factors research has produced a testbed and 
experimental designs for investigating multi-person information processing 
and decision making. The principal independent variables are: team structure 
(hierarchical and parallel); information and communication networks (decentral- 
ized, centralized and partially overlapping) and levels of expertise. This 
component of the program seeks to test and refine the normative mathematical 
models. Pilot studies have been conducted in preparation for a series of 
experiments that will test normative models for information processing and 
resource allocation functions. 

Report ; 

Castanon, D.A., Entin, E.E. & Kleinman, D.L. (1985). Research in distributed 
decision making . Annual Summary Report, Alphatech, Inc. Burlington, MA. 

Archival Publications ; 

Castanon, D.A., Chang, S.C. Luh, P.B., & Wu, Z.J. (1985). Optimal task 
allocation for a team of two decision makers with three classes of impatient 
tasks , 8th MiT/ONR Workshop on 03 systems, Cambridge, MA. 

Serfaty, D. and Kleinman, D.L. (1985). Distributing information and decisions 
in a team . Proceedings of the IEEE Conference on Systems, Man and Cybernetics, 
Tucson, AZ. 
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DISTRIBUTED TACTICAL DECISION MAKING 
NR 649-003 

Massachusetts InstJtute cf Technology /Athans & Levis 



Scientific Oblectives; The objective of the research is to address theoretical 
and computational Issues that arise in the modeling and analysis of 
distributed tactical decision making. The research plan has been organized 
into two highly interrelated research areas: (a) distributed decision 
processes; and (b) distributed organizational structures. 

^PP^°^^^ - Theoretical and experimental investigations are being conducted to 
develop: (a) models and algorithms for decision strategies of distributed 
agents with communications restrictions in situation assessment and resource 
allocation tasks and (b) models and methods for describing and evaluating 
alternative organizational structures including serial, parallel and 
multi-echelon structures. 

Progress: To capture the Interactive Improvement in the solution of 
underlying optimization problems, recent models of individual decision making 
are being extended toward the development of goal decomposition models that 
define the role of the local decision maker, and the limits of his 
authority. A computational model referred to as the performance-workload 
locus, has been developed for evaluating alternat^e organizational 
structures. Models of asynchronous decision process protocols that define 
coordination points, concurrent operations and random task-completion times 
have been formulated using stochastic time Petri Nets. 

Reports : 

Tsitsiklis. J.N. (1984). Problems in decentralized decision ^^kir^o .nH .n,np..r..^». 

Report LIDS-TH-1424. Laboratory for Information and Decision Systems. MTT. 

Cambridge, MA. 

Bejjani. G.J. and Levis. A.H. (1985). Information storage and access in 
decisionmaking Organizations LIDS-P-1466, Laboratory for Information and 
Decision Systems. MIT, Cambridge, MA. 

Boettcher, K.L. (1985). A methodology for the analysis and design of hu man 
information processi ng organizations . Rep LIDS-TH-1 5ni . Tahnra^^r^^r f».- 
Information and Decision Systems. MIT, Cambridge, MA. 



DECENTRALIZED RESOURCE MANAGEMENT 
NR 649-004 

Carnegie-Mellon University /Jensen 



IN TACTICAL COMPUTER EXECUTIVilS 



Scientific Objectives: This research addresses the problem of computer 
system configuration, i.e., the dynamic assignment and reassignment of data 
to processing nodes, at the level of the operating system vhich supports 
multi-person decision making. 
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Approach ; The technical approach entails the de^'elopment of: (a) concurrency 
control theory and mechanisms for decentralized resource management and 
scheduling; and (b) reconfiguration algorithms to meet changing operational 
demands. An important feature of the technical approach is to treat the 
decentralized operating system as a special form of a distributed data base. 
This calls for incorporating within the kernel of the operating system an 
atomic transaction facility. The approach taken is to supplant the 
traditional model or serialization transactions with transactions of greater 
concurrency and modularity. 

Progr ess: A formal theory of modular concurrency control and failure recovery 
has been formulated. The theory is a generalization of classical 
serializability theory and failure atomicity. The rules of the theory 
provide for the consistency of shared data and allow transactions to be 
programmed, modified and scheduled independently of the rest of the 
transactions in the system. In addition, a hybrid system of fast algorithms 
have been developed for dynamic system reconfiguration; thwde algorithms 
operate in near real time and minimize the disturbance to the functioning 
tasks in the course of reconfiguration. 



Report : 

Jenson, E.D. (1985). Decentralized resource management in tactical computer 
executives . Annual Summary Report, Pittsburgh, PA. Carnegie-Mellon 
University, Department of Computer Science. 



THEORY OF INFORMATION PRESENTATION FOR DISTRIBUTED DECISION MAKING 
NR 649-005 

Engineering Research Associates, Inc. /Noble 

Scientific Objectives : Formulate and test a theory of information presentation 
that embodies fundamental elements underlying the understanding and effective 
transfer of complex information among dispersed decision makers. The proposed 
theory focuses on the refinement and integration of schema as methods to 
improve the presentation and understanding of visually displayed information. 
Schema, as defined in this theory, are structures which enable situations to 
be classified and associated with appropriate action rules. 

Approach : The theory predicts the relationship between the features ^amphasized 
in a situation display and people's assessments of different alternatives for 
that situation. The theory suggests how situation fi atures should be tailored 
in order to promote consensus, and what information should be transmitted to 
effect a common situation understanding. The research tests the theory by 
varying situation display features and computing the covariance between measured 
and expected situation classifications, alternative assessments and decision 
conflicts. 

Progress : Methods for constructing and representing schema in graphic and 
symbolic form have been developed. The essential characteristics of schema 
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include: (a) slots for features that define a situation or object; (b) 
constraints that limit features to fill slots; and (c) inferences and action5= 
vhich are appropriate if a schema is activated. Experiments are underway 
using schema representations of enemy fleet deployment scenarios to 
investigate how the level of threat is assessed as a function of the form in 
which information is presented. 

Report ; 

Nobel, D.F. & Truelove, J. A. (1985). Schema-based theory of information 
presentation for distributed decision making Tech Rep. R-028-85. Vienna, VA: 
Engineering Research Associates. 



ACQUISITION AND REPRESENTATION OF KNOWLEDGE FOR DISTRIBUTED COMMAND 
DECISION AIDING 
NR 686-001 

Johns Hopkins University/Hamill & Stewart 



Scientifi c Objective : Investigate critical factors underlying knowledge-based 
decision-aiding systems to support spatially separated tactical commanders in 
Navy command and control environments. 

Approach; The research develops models arJ collects performance data which wil] 
contribute to a theory of distributed decision making. The research focuses on: 
(a) the acquisition of relevant knowled^ froip subject-matter experts; (b) the 
representation of knowledge (c) the utilization of knowledge in problem 
solving and decision -?isks. Algorithmic and heuristic approaches are explored 
in the development r -^tro] structures for the representation of knowledge 
bases. Models of i 'erson decision making are formulated and common 

structures in the . m process investigated. Measures of decision making 

will be developed am xperiments conducted in command and co'Jtrol simulation 
facilities to verify the models and to investigate the features underlying 
knowledge-based systems . 

Progress : A taxonomic structure has been developed to identify and organize 
distributed decision making. The taxonomy is constructed along the three 
dimensions of psychological, computational and communicative processes. Each 
dimension is defined in terms of acquisition, representation and utilization" 
of knowledge. Work has been Initiated on the classification and evaluation of 
knowledge representatior, formalisms, e.g., Franz Lisp, Interlisp-D, YAPS; 
important differences have been observed in capabilities to support semantic 
network structures and mulitiple rule sets. 

Report ; 

Hamill, B. W. & Stewart, R. L. Acquisition and representation of knowledge 
for distributed command decision aiding. Annual Summary Report. Laurel, MD: 
Johns Hopkins University, Applied Physics Laboratory. 
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Archival Publication: 



Hamlll, B. W. (1984). Psychological issues in the design of expert systems 
(p: 73-77). In Proceedings of the Human Factors Society 28th A nnual Meeting . 
Santa Monica, CA: The Human Factors Society. 



MODELS OF GROUP DYNAMICS IN DISTRIBUTED TACTICAL DECISION MAKING 
NR 686-003 

PAR Technology Corp . /Adelman & Lehner 

Scientific Objectives ; The objective is to conduct theoretical and 
experimental research on the psychological determinants of group decision 
making. The focus is on developing a psychological model that is applicable 
to decision environments with a flexible and reconf igurable command and 
control system in a multi-threat environment. 

Approach : The proposed psychologicil model distinguishes between: (a) 
individual and group input factors; (b) characteristics of the group's 
interaction process; and (c) various outcome variables. Issues that are 
investigated within this model include: (1) the extent to which a 
division of responsibility is impacted by alternative communication 
networks and protocols; (2) the degree to which a cooperative vs. non- 
cooperative environment impacts distributed-team performance; and (3) 
the extent to which a common cognitive model of a decision problem in tactical 
situations is required for effective team performance. 

Progress ; A conceptual model has been developed that describes relationships 
and variables postulated as important mediators of group performances in 
distributed decision environments. The model derives primarily from research 
on communication networks, mental models and game theory. The principal 
mediators? in this model are organizational structure affecting division of 
labor and resources, cognitive model similarity and consistency, and norms 
supporting group cooperation. Experimental designs have been developed and 
pilot studies conducted in preparation for a major experiment on resource 
allocation by distributed decision makers. 

Archival Publication ; 

Adelman, L., Zirk, D. & Lehi.er, P.E. (1985). A conceptual framework for 
a ddressing group dynamics in distributed decision making . Proceedings of 1985 
Thee conference on Systems, Man and Cybernetics. 
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BEHAVIORAL ASPECTS OF DISTRIBUTED DECISION-MAKING 
NR 686-004 

Perceptronics/Fischhof f & MacGregor 



Scientific Objective : Develop a behavivora.l ]y-based theory of distributed 
tactical decision-making that extends existing research on individual 
decision-making Into this realm. 

Approach ; Distributed decision systems have been characterized, initially, 
by the number of Individuals or groups Involved in a system or subsystem and, 
subsequently, by behaviorally significant features of the tasks facing those 
In the system. These features are drawn from cognitive psychology, behavioral 
decision theory, and organizational theory, and they include the uncertainty 
surrounding the decisions to be made, the clarity of the systems' overriding 
policy, and the heterogeneity of the participants In the system. 

Progress: A theoretical framework was developed with the properties and 
realities of a variety of existing distributed decision-making systems In 
mind. It was extended to detailed analyses of several specific systems and 
the recurrent problems facing them. Analyses include: (1) the challenges faced 
by systems In avoiding "strategic surprises," wherein their shared model of 
reality becomes Inc reaslngly and profoundly out of touch. (2) the behavioral 
properties of the tasks; and (3) the role of ^yntems whose nature may change 
markedly from their Inception to their operation. A medium fidelity simulation 
of a distributed decision-making system will Investigate at least two Issues: 
how broadly-stated principles of system philosophy are interpreted In specific 
situations and how the distribution of Information within a system influences 
the shared model (s) held by the system participants. 



DISTRIBUTED ROBUST COMMUNICATION CONTROL 
NR 662-004 

University of Maryland /Ephremldes and Makowski 

Scientific Objectives : To Investigate and develop algorithms for radio 
communication networks that can be Implemented in a distributed fashion, for 
the purpose of ultimately integrating the communication issues in the 
distributed tactical decision-making command-and-control framework. 

Approach ; A dual approach of analytical study and algorithm development is 
pursued. An analysis focuses on performance evaluation of communication 
protocols In realistic multlhop networks with variable topology and on 
adaptive control procedures for routing and resource allocation as well as 
on distributed detection/estimation Issues associated with the operation of 
such networks. The algorithm development emphasizes the practical specification 
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of intelligent heuristics for network connectivity establishment, maintenance, 
and utilization, that posses good performance attributes In terms of surviv- 
ability, effectiveness, and efficiency. Both approaches consider the use of 
spread-spectrum signaling that imposes new constraints while at the same time 
opening up new degrees of freedom in the coiranunication control process. 

Progress ; The notion of system dominance was shown to be a powerful tool 
for studying the performance of non-standard queueing systems which naturally 
arise in the modelling of various communication functions. The methodology 
was successfully applied to systems with synchronization constraints, such as 
the Fork-Join queue, where computable bounds and approximations were obtained 
for various statistics of the system response time, both in transient and 
steady-state. A similar approach was taken in the study of interacting 
queues modelling ALOHA-like contention systems. Here, simpler bounding 
systems were identified and stability regions for the original system were 
determined. 

Report ; 

Ephremides, A. & Makowski A.M. (1985). Distributed robust control in radio 
networks. Annual Summary Report. College Park, MD. : University of Maryland, 
Department of Electrical Engineering. 
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E. SPECIAL PROJECTS 



Man-Machine Systems Technology 

This Exploratory Development (6.2) project is designed to extend the 
basic research program In Engineering Psychology toward applications In Navy 
man-machine systems. Currently the project consists of applied research in 
three topic areas: decision making in command and control systems; human- 
factors design for maintainability; and supervisory control. In the command- 
ard-control area, the various work units are investigating information 
processing and decision performance in Nava] mission planning, airborne ASW, 
submarine combat control, and surface ASW. In the maintenance area, efforts 
are directed toward the development of analytical techniques and models of 
technician performance which can be used to predict the maintainability of 
equipments as a consequence of design characteristics. The work on 
supervisory control is exploring the application of new interactive control 
concepts to underwater vehicles and remote manipulators. 

Defense Small Business Advanced Technology Program (DESAT) 

The objectives of the DESAT program Include stimulating technological 
innovation to the private sector, strengthening the role of small business in 
meeting the DoD research and development needs, and increasing the commercial 
application of DoD-supported research and development results. 

The research conducted by Intelligent Software Systems, Inc. (NR DSA-009) 
has as its goal the design, assembly, test, and demonstration of a prototype 
graphics workstation for real-time analysis and extraction of information 
from large data bases. The system will have the capability to generate a 
large set of graphic formats while the user will have the capacity to choose 
a preferred format from a data base and to monitor and dete'^t a set of events 
in a dynamic data-base. These demonstrations will explore a wide range of 
display preferences by a representative group of users for different tasks 
pertaining to process control. The second research effort, conducted bv 
Decision Science Consortium. Inc. (NR DSA-010) , has as its objective the 
design, assembly, test, and demonstration of a prototype system for a 
personalized and prescriptive decision aiding system. This system will 
accommodate to individual differences in preferred levels of problem analysis, 
information search organization, decision rules, and display features. The 
task addressed will be a personnel allocation decision problem. 
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MODELING HUMAN DECISION PROCESSES IN COMMAND AND CONTROL 
NR 460-001 

Alphatech , Inc . /Wohl 



Scientific Objectives ; To develop quantitative models of human decision 
processes in ASW command and control functions. 

Approach ; The approach involves the development of normative-descriptive 
models based on the Stimulus-Hypothesis-Option-Response (SHOR) paradigm. 
The decision functions of the antisubmarine warfare commander (ASWC) have 
been selected as the context for model development. The total ASW 
simulation is comprised of two components; a model of the ASW environment 
and an ASWC tactical decision model. The environmental model simulates the 
dynamics and operational characteristics of the ASW platforms, ASW sensors, 
and enemy submarines. It outputs the sensor data and the own-force platform 
states to the ASWC model. The ASWC model consists of two submodels; (1) a 
model that represents the ASWC's hypothesis generation and evaluation 
procedure (situation assessor); and (2) a model that represents the ASWC's 
option evaluation procedure (resource manager) , 

Progress ; Data encoding and computer programming have been completed. 
Representative inputs to the model include; ASW force composition and 
operational capabilities, likelihood functions for sensor probability of 
detection, enemy submarine trajectories and cognitive style of the 
commander. The model outputs a time series of commands issued by the ASWC 
related to the management of his resources and also provides a number of 
performance measures, such as time between initial contact and prosecution 
and percentage of contacts localized. Experiments are underway to validate 
the situation assessment module. 

Potential Applications ; These models of human decision processes have 
potential for application in the following areas: (1) methodological tool 
for design of C systems; (2) modelling basis for the development of decision 
aids for situation assessment; and (3) simulation methods and models for ASW 
tactical training devices. 



HUMAN FACTORS PRINCIPLES FOR SUBMARINE TACTICAL DECISION AIDING 
NR 460-002 

Decision Science Consortium, Inc/Cohen and Chinnis 



Scientific Objectives ; The generation and testing of human ^actors 
principles for the design of tactical decision aids based on theories 
and models of human decision making. A major objective is the 
determination of the forms of human-comput.^r interaction which are most 
effective in guarding against human information-processing biases and 
limitations. 
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Approach: Experiments are conducted using a computer-based attack planning 
aid which has been implemented in the submarine combat simulation laboratory, 
NUSC, Newport. The experiments involve tests of hypotheses regarding 
cognitive mechanisms and factors that may influence decisions during a 
passive approach against a hostile submarine. These factors include: 
attentional selectivity with regard to risk to own ship, target importance, 
confidence in the process of integrating target range estimates, weighting 
critical probabilities (e.g., of counterdetection) , attempting to eliminate 
uncertainties prior to decision, and framing (e.g., describing potential 
outcomes in terms of hits or misses). Decision aid features to address 
each of these factors are tested. 

Progress : The investigators have developed computer-based interface 
functions which provide both prescriptive and descriptive constraints. These 
include: 1) PLANNING for data entry and query at various levels of specificity 
and aggregation; 2) SELECT EXPLANATION for an examination of inference rules 
and conclusions; 3) ADJUST for making changes in data values and previewing 
the impact of changes on subsequent events; 4) ALERT for prompting when 
thresholds are reached and/or new events occur; :>) ADVISORY for prompting 
when cognitive strategy may be suboptimal and/or when there is evidence of 
conflicting lines of reasoning. 

Potential Applications : The testing of models of cognitive performance in a 
realistic setting with representative participants provides valuable insight 
into the validity of such models, as well as needed guidelines for the design 
of computer-based command and control systems. 

Report : 

Leddo, J. & Cohen, M. (1985). Experimental designs for research on submarine 
tactical decision aidinp (Technical Memorandum). Falls Church, VA. : 
Decision Science Consortium. 



HUMAN FACTORS IN EXPERT SYSTEMS 
NR 460-009 

Naval Ocean Systems Center/Allgaier 



Scientifi c Objectives ; This work unit focuses on human factors issues 
affecting the development and use of computer-based expert systems for Naval 
mission planning functions. It is part of an interdisciplinary effort 
directed at the development of human factors and computer science 
technologies for improved decision-support systems. 

Approach: The human factors effort concentrates on four technology issues: 
(1) definition of functional requirements and development of decision-process 
models of mission planners; (2) reliable methods for knowledge elicitation 
and representation; (3) development of a man-machine interface and 
explanation facility; and (4) performance assessment. The air strike 
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planning function was selected as the context for this work which included 
the development of an experimenta] expert-system for mission planning. 



Progress ; The underlying structure and principal components of the 
experimental expert-system have been defined, A task analysis and decision 
process model for air strike planning functions were developed with the 
assistance of operational personnel. Existing strike-planning decision aids 
were assessed with a view to identifying relevant heuristics and algorithms. 
Efforts are directed to the development of a module for planning weapons 
loading; subsequent work includes modules for weapons selection and route 
planning functions. Menu-based interface concepts, based on CMU-ZOG develop- 
ments are explored and a Lisp object-oriented programming system was selected 
for investigations of methods for knowledge representation. 

Potential Applications ; The results of this work will contribute to the 
human factors and computer science te':hnology base by providing improved 
methods and priniciples for use in the development of computer-based expert 
systems. These technological developments have the potential for signifi- 
cantly enhancing human decision performance in tactical planning functions. 

Report ; 

Ehler, P, Osga, G,, Lambert, D, & Cook, T. (1985). Expert systems in support 
of composite warfare commander mission planning (Tech, Rep. 1067), San Diego, 
CA; Naval Ocean Systems Center. 



HUMAN FACTORS AFFECTING MAINTAINABILITY OF SHIPBOARD PROPULSION SYSTEMS 
NR 503-001 

Perceptronlcs, Inc. /Madni 



Scientific Objectives ; (1) To develop improved methods for representing and 
analyzing complex maintenance tasks in shipboard propulsion systems; and (2) 
to develop and evaluate concepts and principles for the human-engineering 
design of equipments and maintenance suppor*: systems in order to improve the 
maintainability of shipboard propulsion systems. 

Approach ; A model-based approach for representing and analyzing maintenance 
tasks was developed. The approach involves modelling human actions, decisions, 
and responses to specific events within Modified Petri-net representations. 
Within the framework it was possible to; (a) identify procedural inconsis- 
tencJes and ambiguities that impair human performance; (b) predict task-related 
workload; (c) develop guidelines for evaluating human-equipment interface 
options; and (d) determine where task reallocation was necessary and feasible. 
Operator tasks related to the maintainability of shipboard gas turbine 
propulsion systems were selected as the context for the modeling work which 
includes individual and cooperative tasks as well as multiple equipment 
malfunctions. 
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Progress; A knowledge-based task analysis framework using the Modified 
Petrinet model was developed to identify potential man-machine interface 
problems. Model-guided expert elicitation and model refinement was 
performed to explicitly encode contingency handling procedures. The 
model was implemented to exhibit dynamic task execution and on-line 
performance /workload predictions based on the elicited procedures and 
expert ratings. Key model elements were verified with data collected 
from gas turbine training simulation exercises. 

Potential Applications ; The modeling techniques and computer-aided analysis 
tools have the potential for identifying improved man-machine interface design 
options, task allocation, and high pay-off job-aiding functions. 



COMPUTER-BASED MODELS FOR MAINTAINABILITY DESIGN 
NR 503-003 

University of Southern Cal if ornia/Towne 



Scientifi c Objectives ; The objective is to develop models for predicting 
equipment maintainability as a consequence of its internal structure and the 
design of the man-machine interface. A central feature of the work is the 
development of a computer-based model of maintenance-technician performance 
including fault uiagnosis and repair. 

Approach; A computer-based model referred to as PROFILE was developed 
for assessing the maintainability of a system. The model uses data 
representing the system's physical construction, internal organization and 
fault-effects. For each fault of interest, PROFILE generates a sequence of 
actions required to isolate and correct a malfunction, and it determines the 
time to perform the generated procedure by accessing a data bank of standard 
times for generic maintenance actions. 

Progress; Detailed performance data derived from experiments were analyzed 
to determine the variables which affect the cognitive rime to accomplish 
fault diagnosis. The PROFILE model was enlarged and refined, to project the 
cognitive component as well as the manual component of equipment maintenance. 
With this expansion, and the completion of the graphical interface to PROFILE, 
the system was tested for the accuracy and sensitivity of the technique, as 
well as the ease of use by design personnel. 

Potential Applications ; The model has application within the design 
cycle to aid the product engineering process. It can assist in evaluating 
the maintainability characteristics of completed design proposals. 
Researchers in human factors and design may employ the model to 
explore relationships between design and maintainability. 
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Report ; 



Towne, D. M. & Johnson, M, C. (1985). Cognitive workload and complexity in 
fault diagnosis (Tech. Rep. ONR-106). Los Angeles, CA; University of 
Southern California. 



REMOTE CONTROT. OF UNDERWATER VEHICLES AND MANIPULATORS 
NR 512-001 

Massachusetts Institute of Technology/Sheridan & Yoerger 



S cientific Objectives ; The objective of this work is to investigate 
man-machine features and control models for deep-ocean, remotely operated 
vehicles and manipulators. Computer-aided supervisory control techniques 
are being evaluated to improve operator performance and overall system 
capabilities through concurrent control of both the vehicle and the 
manipulators. 

Approach ; The approach taken combines engineering analysis, simulation, and 
experimentation. A task analysis is being performed based on operational 
experience in deep ocean tasks. Dynamic models of a vehicle and its 
manipulators are being used to evaluate alternative designs. New nonlinear 
control-theoretic techniques are being used to design a closed-loop control 
system that permits an advanced supervisory architecture to be implemented. 
Analysis and experiments in supervisory control techniques performed under 
related programs at MIT will be used as the basis for the overall control 
system design. 

Pr ogress : Basic functional requirements for the vehicle anc* manipulators 
have been established. Kinematic and dynamic models were formulated and a 
new nonlinear control technique developed using an approach appropriate to 
a wide range of vehicles. Simulation results showed the method can handle 
extreme nonlinearities; the method also contained interesting monitoring 
features that fit smoothly into a supervisory control scheme. Models for 
concurrent control of a vehicle and manipulators and the design of a new 
master controller are being investigated. 

Potential Applications ; The supervisory control concepts and models will 
have application to the design of advanced Navy underwater search-and- 
inspection vehicles. 
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c!Sv?2^^^^^ INTERACTIVE DISPLAY AND ANALYSIS SYSTEM FOR INDIVIDUAL DECISION 
STYLE 

NR DSA-010 

Decision Science Consortium, Inc. /Cohen & Chinnis 



Scientific Objectives; Decision-making style for a widt> range nf users is 
defined and measured within the context of a decision-support system User 
accommodation to analytic decision-aids that provide guidance against bias or 
error is assessed. Principles are defined and expressed within independent 
software modules that provide flexible interfaces with the user that are 
consistent with individual decision-making styles and facilitate those 
strategies. 

Approach; Decision aids are developed that accept the user's personal 
preferences but through channeling and prompting suggesc optimal decision 
strategies consistent with those preferences. Individual differences in 
method of problem structuring and analysis, organization, and searching a 
data base, are accommodated for styles that range from the intuitive to the 
analytical. Coupled with this capability, the decision aid monitors the 
user s decision strategy and suggests potential biases or errors that are 
evident in the user's decision-making. The decision-support system consists 
of a data base for the task, an analytic model of the task, a planning module 
for interacting with the data base, and other software modules that allow the 
user to select tha relevant parts of the data base, to insert judgments, to 
evaluate I... e options presented, to monitor conclusions from the analytic model, 
and to alert the user when a decision differs significantly from norms. 

Progress: A prototype aid was developed and tested successfully for tactical 
decisions made by submarine commanders relative to the launching of an atta-k 
against a target. The aid has currently been extended ::o the more general 
situation of a manager's selection of a new employee among a number of 
applicants. This latter decision aid has been designed in six modules: (1) 
knowledge base - decision task and its context; (2) adjust - user changes the 
knowledge base; (3) decide - user specifies strategies and options; (4) 
select - user manipulates subsets of the knowledge base; (5) alert - system 
prompts user about significant options; and (6) advisory - system prompts 
user about suboptimal strategies. The implementation of the decision aid has 
been partially achieved on an IBM-AT systen in the C language. 

Potential Applications: The personalized/prescriptive design principles for 
an interface to a decision-support system have been demonstrated to be 
effective In tactical situations and a multi-attribute choice situation. The 
generality of this system sugg.^sts that such decision aids would ba suitable 
for planning, policy options, and management decision-making 



ERIC 



46 

5'} 



****** 
THE FOLLOWING WORK UNIT WAS COMPLETED DURING THE PAST YEAR: 



FLEXIBLE GRAPHICS WORKSTATION FOR DECISION MAKING 
NR DSA-009 

Visual Intelligence Corporation/Morse 

Scie ntific Objectives ; Define and vallc'ate cognitive-interface principles 
during interactiye experiments on a workstation providing graphic presentations 
of complex, time-varying and dynamic data. 

Approach ; A prototype workstation was designed, assembled and tested with 
experienced users who performed real-time analysis and extraction of 
information from large data bases. A representative task environment was 
simulated that encompassed problems that surround real-time decision making. 
A series of experiments validated design principles for the user Interface of 
that prototype workstation under these conditions: (a) simultaneous and /or 
sequential display of data in multiple formats; (b) embedding of static 
data-displays in user-created backgrounds; (c) dynamic formatting of data 
displays; (d) introduction of event monitors as an aid to user strategy in 
reacting to real-time data; and (e) the affect of total-system context on 
user perfomance. A series of experiments validated the display principles 
for the user interface by formal evaluation with novice users as test 
participants. 

Results and Conclusions: Software modules were designed and written for the 
graphics display system; they included modules for: (a) a user Interface to 
system functions; (b) an interactive facility for the construction of display 
formats; (c) a computer-aided design aid for the construction of background 
contexts; (d) an intelligent event monitor; and (e) a data system interface. 
Formal tesring of the system with novice users demonstrated that the design 
principles incorporated into the workstation facilitated the encoding, 
retrieval, and modification of data, as well as t ;e monitoring of significant 
changes in large data bases. 

Pote ntial Applications : In Navy information handling systems where the 
is an opevational requirement at various user levels to reach quick decisions 
based on the review of large amounts of information, interactive workstations 
will be effective. Graphic displays, in flexible formats chosen by the user, 
stored in knowledge bases, and triggered by various event conditions, show 
great promise to achieve needed capability. 
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GROUP PSYCHOLOGY RESEARCH PROGRAMS 
(CLUSTER F-I) 

From 1973 to 1984 this program was called "Organizational Effectiveness" 
and aimed at understanding the variables that determine the job performance, 
morale, retention, and quality of working life of members of work organiza- 
tions. During that period we funded a number of field studies dealing with 
organizational theory and organizational behavior. At the end of 1984 we 
changed our name to "Group Psychology," and decided to phase out our organiza- 
tional research. We intended to focus on formal models and theory-guided 
controlled experimentation on performance in small groups ~ especially those 
which work unaer stressful conditions or are hierarchically organized 
During this year, ONR management has reassessed the total range of life sciences 
wnich it desires to support, and has concluded that Group Psychology cannot be 
assigned a high priority. In FY 1987 this domain will not appear as an ONR 
program. 

C-.r research during the period covered by this volume consisted of three 

Re^ent[nn'^"''T«'^ ''t^°V "^^^^^^^^ Heterogeneous Groups," G. "Turnover and 
Retention," and H. Productivity." 

Cluster I contains contracts belonging to an Exploratory Development (6 2) 
program on Manpower R&D, managed by Dr. Stan Collyer in ONR's Technology Programs 
Directorate. It is multidisciplinary, with specific contracts monitored by 
Scientific Officers in both the Psychology Sciences and Mathematical Sciences 
Divisions. Within the Psychological Sciences Division, the Manager of the 
Group Psychology Research Programs serves as Manager. 
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F. EFFECTIVE HETEROGENEOUS GROUPS 



ONR-sponsored research on intergroup relations and the functioning of 
heterogeneous groups began in the early 1970s when the Armed Services dropped 
conscription and increased their recruitment of minorities and women (and 
when a number of well-published racial incidents occurred in the Navy and 
Marine Corps). ONR's initial efforts, designed to increase the knowledge base 
needed to improve intergroup relations, included laboratory and field studies 
on the dynamics of interactions among persons from different cultural, ethnic, 
and racial groups. 

In the late 1970s and the 1980s, our research in this area has con- 
centrated on identifying the situations and contexts that contribute to the 
effective functioning of heterogeneous groups in high-technology organi- 
zations. A number of efforts aimed at deepening our understanding of the 
factors affecting intergroup and interpersonal behavior. They included 
research on cognitive organization of information about persons from different 
groups, determinants of attitudes toward different ethnic and racial groups 
and the effects of organizational membership on intergroup behavior. Several 
contracts explicitly focused on the economic, social, psychological, cultural, 
and vocational variables critical to successful recruitment, education, 
training, man gement, and retention of minorities in high- technology 
occupations. This research, when completed over the next year or two, will 
provide information about organizational contexts, which when combined with what 
we learned previously about the perceptions and motives of minorf.ties and the 
behavioral responses of minority and majority persons to each other, will 
provide a substantially improved technology base. 

At this point, therefore, it is appropriate to terminate our financial 
support of research in this area and provide an opportunity for the Navy's 
applied R&D and operational manpower programs to transition the products of 
our past basic research. As a matter of fact, the Navy has already made 
substantial progress in applying the results of such past psychological 
research in the form of the Navy Organizational Effectiveness Program 
(formerly the Navy Human Resources Management Program) managed by the Deputy 
Chief of Naval Operations (Manpower, Personnel, and Training). In addition, 
the current success of the Navy in recruiting and retaining minorities in the 
enlisted ranks suggests that a number of management lessons have already been 
learned that might well be applied now to recruitment of minority officers, 
where there is still room for improvement. All in all, this seems an 
opportune time to turn our basic research attention to other areas where the 
technology base still needs strengthening and/or where exciting discoveries 
are being made. 
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THE USE OF COOPERATIVE LEARNING PROCEDURES WITH HETEROGENEOUS TEAMS IN HIGH- 
TECHNOLOGY ORGANIZATIONS 
NR 170-967 

University of Minnesota/Johnson & Johnson 



Scientific Ob.lectlves; To develop and refine a theoretical model concerning 
the Internal dynamics and processes of cooperative learning groups. To 
conduct experimental research on the use of cooperative learning groups 
with the learning of technical material at the computer. 

Approach; The basic field-experimental design for this research program 
involves young adults from secondary schools and from ROTC programs on 
two campuses. Students are randomly assigned to an individualistic 
condition, where each student works alone toward a set criteria, a 
competitive condition where each student works to do better than classmates, 
or one or more cooperative conditions where students work together in small 
squads to make sure all of them have learned the material. A range of 
different cooperative conditions are being investigated with three studies 
being conducted each year. Some of the internal dynamics of student/student 
interaction being examined Include: different goal structures (cooperative, 
competitive, and individualistic), levels of cognitive reasoning, mode of 
participation, and purpose of the learning. 

Progress ; Study one used a computer program on navigational sklT":- with 
secondary school students. Students in the cooperative/computer condition 
scored higher on the achievement and attitude measures than did students in a 
competitive/computer condition or an individualistic/computer condition. There 
seem to be some advantages to structuring learning teams at a computer even if 
there are enough computers to send students to a computer alone. Study two 
examined the effects of controversy in a cooperative condition. The 
university-level engineering students in the controversy condition produced 
significantly better final reports and more positive attitudes toward 
controversy than classmates in the no-controversy condition while the students 
in the no-controversy condition were more positive about the subject matter. 

Potential Applications ; This research will provide specific strategies for 
structuring cooperative learning groups to maximize the learning of technical 
material in Navy and other high-technology training programs. 

Reports ; 

Johnson, R. T., Johnson, D. W. , & Stanne, M. B. The Effects of Cooperative. 
Competitive, and I ndividualistic Goal Structures on Computer-Assiste d 
Instruction (TR-ONR-2) . University of Minnesota, Dept. of Ed. Psych. 

Smith, K. A., Petersen, R. P., Johnson, D. W. , Johnson, R. T. The Effects of 
Controver sy and Concurrence Seeking on Effective Decision Making (TR-ONR-3) . 
University of Minnesota, Dept. of Ed. Psych. 

Yager, S., Johnson, R. T., Johnson, D. W. The Impact of Group Processing on 
Achieveme nt in Cooperative Learning Groups (TR-ONR-4). University of 
Minnesota, Dept. of Ed. Psych. 
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Yager, S., Johnson, R. T. , Johnson, D. W. Oral Discussion , Group-Individual 
Transfer and Achievement in Cooperative Learning Groups (TR-ONR-5). 
University of Minnesota, Dept. of Ed. Psych. 

Johnson, R. T., Johnson, D. W., Stanne, M. B. A Comparison of Computer-Assisted 
Coperative, Competitive, and Individualistic Learning (TR-ONR-6) . University 
of Minnesota, Dept. of Ed. Psych. 

Johnson, D. W. , Johnson, R. T. Computer-Assisted Cooperative Learning (TR-ONR- 
7). University of Minnesota, Dept. of Ed. Psych. 

Lew, M., Mesch, D., Johnson, D. W. , Johnson, R. T. Effects of Collaborative- 
Learning: Contingencies on the Social Interaction of Isolated Students (TR- 
ONR-8) . University of Minnesota, Dept. of Ed. Psych. 

Lew, M. , Mesch, D., Johnson, D. W. , Johnson, R. T. Positive Interdependence, 
Academic and Collaborativa-Skills Group Contingencies and Isolated Students 
(TR-ONR-9). University of Minnesota, Dept. of Ed. Psych. 



CAREER ATTAINMENT FOR WOl'IEN, BLACKS AND HISPANICS IN CIVILIAN AND MILITARY 
HIGH-TECHNOLOGY ORGANIZATIONS 
NR 170-968 

Johns Hopkins University/Braddock & Grain 



Scientific Objectives ; This research is testing social status attainment 
hypotheses which seek to explain the process of identifying, recruiting, and 
training blacks, hispanics and women across a wide range of high-technology 
occupations. Private firms, public agencies, and military commands of 
different sizes and in different locations are being investigated. 

Approach : The National Longitudinal study of the High School Class of 1972 
developed by the National Center for Education Statistics has been enhanced by 
information from public and private employers on practices related to the jobs 
held by these young adults, and by data from Department of Defense sources on 
the military careers of these individuals. This work differs from most 
previous research by combining individual variables, such as educational 
background, with labor market variables, such as employer practices, and by 
studying various stages of the employment process (recruitment, selection, 
training, turnover or promotion). 

Progress : The early stages of this proje».t were devoted to building 
a data base which includes Information on firms and local labor markets as wel 
as individuals. Census data on local labor markets and new data from a 
nationwide survey of A078 employers with data from a seven year follow-up of 
the National Longitudinal Study have been merged. A critical review of the 
literature and some preliminary findings relating recruitment practices to 
race and gender representation in the work force were presented at ONR 
conferences in 1983, and subsequently released as a Technical Report in 
1984. Presentations based on current findings were presented at the American 
Sociological Association and the American Educational Research Association 
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during 1985 and will be released in a technical report during 1986. 



Potential Applications ; Any organization, whether a school, a firm, or a 
military command, that wants to increase the representation of minorities or 
women in high-technology fields must consider at what point (s) it can 
intervene most effectively in the employment process and how. This research 
will provide a theoretical basis for such decisions. 

Ret orts : 

Braddock, J. Recruitment and selection of minorities in high-tech 
organizati ons; A sociological perspective (TR-ONR-1), The Johns Hopkins 
University, Center for the Social Organization of Schools, 1985 (ADA 151-547). 

INFERENCE PROCESSES IN ATTRIBUTION AND ATTITUDE FORMATION 
NR 170-969 

Purdue University/Smith 



Scientific Objectives ; Test and extend a new theory of social inference 
processes, including those involved in person perception, causal attribution, 
and attitude formation. The theory is based on a general theory of memory and 
cognitive processes; thus, the overall research objective is to create closer 
theoretical linkages between social cognition and cognitive psychology. 

Approach ; Laboratory experiments and analysis of non-experimental (survey) 
data focus on the following issues related to the theory; the automatization 
of social inference processes, people's ability to verbally report on their 
use of inference procedures, the effects of salience on the types of 
inferences made, and the implications of inference processes for the 
long-term memory structure of an attitude. 

Progress ; Studies have been conducted on the following basic issues. (1) 
The development of automatic procedures for social judgments, including 
increases in speed and efficiency of judgments with practice. Consistent 
with the theory, results show that as they become automatized social 
judgment processes display changes that parallel those that have been found with 
nonsocial judgments. (2) The role of procedural knowledge in category 
accessibility effects; procedural knowledge, an important component of the 
theory, has been shown to be implicated in long-lasting effects of exposure 
to information lelated to a social category (such as hostility) on later 
interpretations of people's behaviors. (3) The structure of knowledge about 
persons — incuding stereotypes—in memory; Knowledge about other persons is 
organized in memory around concepts of traits, roles, and goals, and can be 
accessed efficiently to make inferences about other people. 



Potential Applications ; The theory of social inference, and the research 
supporting it, have potential applications to stereotyping, person 
perception, and intergroup attitudes. The processes underlying stereotyped 
inferences about people's personality characteristics or ability based on 
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their race, gender, or ethnicity will be clarified by the research, and 
possible means of avoiding stereotype-based biases in judgment may be 
pointed out. 



Reports ; 

Smith, E. (1985). The development of automatism of social judgments 
(TR-ONR-4). Purdue University, Psychology Department. 



****** 
THE FOLLOWING WORK UNITS WERE COMPLETED DURING THE PAST YEAR: 



AN EXAMINATION OF HISPANICS AND GENERAL POPULATION PERCEPTIONS OF 
ORGANIZATIONAL ENVIRONMENTS 
NR 170-906 

University of Illinois/Triandis 

Scientific Objectives ; To compare the subjective cultures of Hispanics and 
samples of the general population, using both anthropological and 
psychological approaches as well as a hybrid approach developed by Triandis. 
To suggest how subjective culture information can be used to improve 
recruitment, training, and retention of Hispanics in high technology 
organizations. 

Approach ; Intensive interviews and observations in Navy recruitment and 
training settings resulted in hypotheses about significant differences in 
the subjective cultures of Hispanics and Non-Hispanics. Instruments 
developed to test these hypotheses included both emic (unique to Hispanics) 
and etic (common to the two populations) elements. Findings obtained with 
Navy personnel were cross-validated with Hispanic and Non-Hispanic high school 
students and against data from the National Longitudinal Survey. The 
methodologies included idiographic analyses, the determination of the 
acculturation level of each individual, the use of multidimensional scaling, 
and factor analyses of the responses ob'zained from each sample. 

Results and Conclusions ; Theor'Jtical results centered around a better 
understanding of the constructs of collectivism (which is an important element 
of Hispanic culture) and individualism (which is an important element of 
Non-Hispanic culture). In addition, one study made a contribution to the 
debate of "What is culture?", and how it should be conceptualized for optimum 
progress in the social sciences. Methodological results centered around the 
issue of whether or not one obtains substantial advantages by developing 
locally sensitive (emic) instruments for the study of cultural differences. 
It was found that while emic instruments are expensive to develop they do 
identify more cultural differences than etic instruments. On the other hand, 
very large cultural differences can be identified with even the crudest 
pseudo-etic instruments. Numerous papers were published that discussed 
the question of equivalence of measurement in cross cultural research. 
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One of the studies examined the behavior of blHnguals responding to counter- 
balanced forms of the same questionnaire and tested the hypothesis that 
social desirability influences the responses of the bilingual in the 
language of the culture vith which the bilingual does not Identify very 
strongly. Another methodological issue is how to handle acculturation in 
cross-cultural studies. It was shown that acculturation influences the 
obtained results differently for different topics: Elements ol subjective 
culture closely linked to behavior converge to the responses of the group that 
a minority is acculturating to, but elements that are not closely linked to 
behavior do not show such convergence. 

Potential Applications : An understanding of the characteristics of people 
from the collectivist Hispanic culture could improve their recruitment, 
training and retention in individualistic high-technology cultures such as 
the Navy. 

Reports : 

Triandis, H.C. (1985). An examination of Hispanic and general population 
perceptio ns of organization environments: Final Report . University of 
Illinois, Psychology Dept., August 1985. (A158672) 

Archival Publications: 



Triandis, H. C. et al. (1984). Simpatia as a cultural script of Hispanics. 
Journal of Personality and Social Psychology , /jT^, 1363-1375. 

Triandis, H. C. (1984). A theoretical framework for the more efficient 
construction of culture assimilators . International Journal of Intergroup 
Relations , 8^, 301-330. ^ — ^ 

Hui, C. H., & Triandis, H. C. (1985). Measurement in cross-cultural 
psychology: A review and comparison of strategies. Journal of C ross- 
Cultural Psychology , 16^, 131-152. 

CHANGING RACE RELATIONS IN MANAGEMENT 
NR 170-943 

Yale University /Alderfer 

Scientific Objectives: To assess the consequences of long-term intergroup 
intervention for achieving attitudJ.al and structural change in the relations 
between black and white managers; to examine and explain the underlying 
dynamics associated with change and resistance to change in race relations in 
large organizations. 

Approach; In contrast to the existing paradigm from social psychology based on 
the contact hypothesis," the present framework utilized concepts from embedded 
intergroup relations theory. The intervention methodology after diagnosis 
consists of three major components: (1) a twenty-person race-relations advisory 
group balanced by race, gender, rank, and department; (2) an upward-mobility 
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program designed to select managers for potential advancement, independently of, 
and in coordination with, normal promotion procedures; and (3) a race-relations 
competence workshop created to teach black and white managers the concepts 
and behaviors necessary to Improve the overall race relations of the 
organization. Assessment of the effects of the interventions was done 
by means of a variety of measures including the products of the advisory 
committee, the qualities and race-gender mix of the people selected for 
advancement potential, the change in proportions of black managers in higher 
level positions throughout the corporation, the participation of black 
managers in decision-making about the careers of lower ranking employees, and 
the concepts and attitudes of individuals who participate in the race 
relations workshops. This project occurred in an organization with more than 
12,000 employees, 3000 managers, and 200 black managers. 

Results and Conclusions ; An historical study of the Race Relations Advisoiry 
Group identified phases in the life of the group and related the development 
periods to group products, internal relationships among members, and external 
transactions between the advisory group and key units from the organizations. 
Intervention procedures for the corporation's upward mobility system were derived 
from the theory. Results of the intervention were examined at each phase of the 
program and demonstrated the effects of both progressive and conservative forces 
in response to the change program. The Race Relations Competence Workshop was 
designed in accord with the theory, provided opportunities for managers to 
learn how to manage racial Cynamics more effectively, and provided further 
insights into the collusive patterns of interpersonal relationships within 
and between the racial groups. Systematic evaluation data suggest that 
blacks evaluated the workshop more favorably than whites, and whites 
increased their awareness of racist behavior patterns more than blacks. 

Potential Applications : The method and theory used in this study have 
potential applicability in any large predominantly white organization that 
has a. significant number of black members, a professional human resource 
program, and a commitment from senior officials to support progressive race 
relations. Advisory group composition, mobility management, and workshop 
design all lend themselves to being adapted to the special conditions of the 
Navy, if the other conditions can be met by specific organizational units. 

Reports ; 

Alderfer, Clayton P. (1985). Introduction to Changing Race Relations in 
Management (Technical Report 3). New Haven: Yele University, School of 
Organization & Management. 

Alderfer, Clayton P. (1985). Changing Race Relations in Organizatons: A 
Comparison (Technical Report 4). New Haven: Yale University, School of 
Organization & Management. 

Alderfer, Clayton P. (1985). The Race Relations Advisory Group: An Intergroup 
Intervention (Technical Report 5), New Haven: Yale University, School of 
Organization & Management. 
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Alderfer, Clayton P. and Tucker, Robert C. (1985). M easuring Managerial 
Potential and Intervening to Improve the Racial Equity of Upward Mobility 
Decisions . (Technical Report 6). New Haven: Yale University, School of 
Organization & Management. 

Alderfer, Clayton P., Alderfer, Charleen J., Tucker, Robert C, and Tucker, 
Leota M. (1985). The Race Relations Competence Workshop; An Intergroup 
Educational Procedure (Technical Report 7). New Haven: Yale University, School of 
of Organization & Management. 

Alderfer, Clayton P. (1985). Time Series Evaluation of Race Relations 
Improvement (Technical Report 8)"! New Haven: Yale University, School of 
Organization & Management. 

Alderfer, Clayton P. (1986). Comparison of Perceptions About Race Relations 
Before and After Long Term Intergroup Intervention (Technical Report 9). 
New Haven: Yale University, School of Organization & Management. 

Archival Publications : 

Alderfer, C. P. (1986). An integrated perspective on group dynamics. In Jay 
Forsch (Ed.), Handbook of Organizational Behavior . Englewood Cliffs, N.J : 
Prentice Hall. 
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G. PERSONNEL TURNOVER AND RETENTION 



Investigators in this cluster have been examining the important variables 
related to: individual characteristics and attitudes (such as organizational 
commitment) which predict personnel turnover; the manner in which features of 
the organization (e.g., leadership and management) are related to turnover; 
the impact of turnover on productivity and organizational effectiveness; and 
the discovery of new methods to reduce turnover. 

A simple measure of the number of individuals retained by an organization 
cannot be used as the sole index of the effectiveness of the organization. 
Instead, one must look at the extent to which high performing individuals 
are not only retained but allowed to develop in a setting which fosters 
positive work attitudes. These attitudes, in turn, can be a critical link in 
reducing personnel turnover and increasing productivity. Our research in 
this area has been characterized by tlie creation and testing of longitudinal 
and multivariate models, and has been concerned with the influence on turnover 
of personnel policies and incentive plans, changes in job s^'ope, and the role 
of til supervisor. In a broader career development approach, we have funded 
investigations of the role of family factors in Navy reenlistment , and the 
interaction between organizational socialization practices and individual 
career decision making. 

Two factors demand terminating our support of research in this area. 
First, we have recently completed a number of major efforts directed at 
personnel turnover which have produced theories, models and instruments which 
considerably improve '^ur ability to predict the extent of turnover, influence 
its incidence, and accommodate to its effects. Two recent books (Employee- 
Organizational Linkages by Mowday, Porter, and Steers and Employee Turnover 
- Causes, Consequences, and Control by Mobley) summarize much of the recent 
ONR turnover research and document rather substantial advances in this field 
of research. Second, the Chief of Naval Operations announced in September 
1984 that retention of Navy personnel has reached an all-time high of 60% of 
those in their first enlistment, 67% of those in the second enlistment, and 
95% of those beyond the second enlistment. Also, recruiting has improved to 
such an extent that 93% of the new enlistees have high school diplomas. This 
represents a major change since the advent of the all-volunteer Navy (which 
is when we began increasing our research on recruiting and retention). Since 
improved recruitment and retention programs do not have the urgent priority 
they warranted in the 1970s, it is appropriate for us to turn our research 
attention to other, more critical areas. 
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THE IMPACT OF FAMILY STRUCTURE ON THE STRUCTURE OF WOFK 
NR 170-956 

Northwestern University/Yogev & Brett 



Scientifi c Oblectives ; Specific objectives include the identification of 
the job behaviors and family-oriented behaviors that conflict, the identification 
of occupational cultures in which work behaviors are changing and the develop- 
ment of psychometrically sound techniques for measuring changes in work behaviors. 

Approach; The investigators developed new instruments, models and analytic 
techniques. New instruments include a family-involvement scale and measures 
of the degree of deviation from work norms about hours of work, place of work, 
social, travel, relocation, and physical energy-health. These deviation 
measures are the dependent variables. Independent variables are: organi- 
zation culture, labor economic conditions, job involvement, family 
involvement and career stage. The model to be tested is at t> couples 
level of analysis; that is, interest is in the causal relaf 5iilps 
between spouses' work-role restructuring. Structural equatl^a modeling is 
used to test the model. 

Progress: In the main study, we are studying couples who fit into the following 
patterns developed in the pilot research: both high work-involved; he high 
work-involved, she low; and vice versa. Working with several Chicago 
area organizations, a target group of employees was located and interviewed 
about the norms of widely held beliefs about what behaviors are necessary to 
get ahead and/ or be seen as an effective performer in that company. The 
interview was followed by a questionnaire to couples in each organization. 
Questionnaire data are used to test the model. 

Potential Applications: The major use of the results of this research is in 
strategic human resource planning. As the workforce absorbs more and mora 
working women with children, organizations are going to have to change 
corporate norms in order to keep these women and/or their husbands who also 
have family responsibilities. Our studies will identify conditions under 
which norms are violated and provide a cross-fertilization of ideas about 
building-in flexible structures. 

Archival Publications 

Yogev, F. (1985). Perceptions of the division of housework and childcare 

and marital satisfaction. Journal of Marriage and th e Family, A7, 3, 609-618. 

(AD A131761). — 
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REACTIONS TO JOB CHANGES 
NR 170-959 

University of Wisconsin /Dunham 



Scientific Objectives ; To assess worker focus of attention (on/off the job), 
absorption with or distraction from the job, and support for /resistance to 
organizational change. To assess the impact of existing job differences 
versus job changes. To develop a system for anticipating and managing the 
climate for organizational change. To explore the processes involved in the 
development of organizational commitinent from the time of organizational 
entry. 

Approach ; Data are being collected through a series of field surveys, 
organizational simulation experiments, and field experiments. Subjects 
include paid subjects and volunteer organizational members (from low-level 
hourly employees through upper-level management). Statistical techriques 
include multiple regression, analysis of variance, multivariate analysis of 
variance, and factor analysis. Innovative approaches include a new content 
analysis procedure, and development of new measures of important constructs 
previously not measured effectively (such as absorption/distraction, 
orientation toward change, etc.) 

Progress ; Primary research emphasis in this year has been on the development 
and examination of a new construct called "focus of attentior." At a basic 
level focus of attention is defined as what a person thinks about while at 
work, including both the direction and intensity of the thoughts. Through 
data collection at six different organizations, focus of attention has been 
measured, the construct validated, and several hypotheses tested. Its 
importance for understanding worker reactions is clear. It has been shown 
to influence reactions to variations in job design, leader behavior, and 
work-unit structure. A second line of effort has identified several 
attitudes underlying attitude toward change, an instrument has been 
developed, data collected, and initial validation tests have been conducted. 
A third effort has examined worker reactions to job redesign, and the 
importance of attitude toward change has been documented in a series of job 
redesign interventions. 

Potential Applications : Diagnosis of probable support for/resistance to 
specific organizational changes. Assessment of degree to which 
organizational members are focusing on job versus non-job factors and the 
impact of this on organizational policies. Better understanding of the 
process by which conmiitment to the organization is developed. 
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THE FOLLOWING WORK UNITS WERE COMPLETED DURING THE PAST YEAR: 

THE EFFECTS OF ORGANIZATIONAL SOCIALIZATION ON CAREER OUTCOMES 
NR 170-911 

M.I.T. Sloan School of Management/Schein, Ballyn, Van Maanen 

Scientific Objectives ; To explore the Interaction between the individual and 
the organization, investigating the effects of culture, socialization 
processes, and human resource policies to determine their effects on 
organizational functioning and individual satisfaction. 

Approach ; This project was a multi-method project focusing primarily on 
ethnographic, clinical, and exploratory survey methods. The variables needing 
investigation are not well understood and no ready-made instruments exist to 
study them. Approaches were chosen to lead to understanding in greater depth 
the meaning of such concepts as culture, socialization, organization, 
individual career, etc. The investigators also decided to study a tew 
organizations and occupations in depth rather than doing extensive statistical 
analyses. 

Results and Conclusions ; At the outset we found that much more effort had to 
be devoted to defining and delineating key concepts such as culture. In fact, 
as the final report shows, we still have a way to go before we can unambiguously 
def '.ne what culture is, much less how it evolves, changes, and affects other 
things such as careers. Human resource policies are typically built on 
oversimplistic assumptions abut career motivations, leading to some 
counterproductive and paradoxical outcomes, especially In technically based 
careers and organizations. Finally, we found that organizations are more 
complex than we had initially thought, requiring more qualitative, clinical and 
ethnographic research methods. In using these methods we also found that 
research in these areas was itself in need of definition and refinement, and 
devoted a technical report to research methods. 

Potential Applications ; The findings will have implications for personnel 
and managerial policy with regard to the management of technical and 
managerial personnel. 

Reports ; 

Dougherty, Deborah (1985). Culture, Socialization, and Careers ; The Individual 
and the O rganization (Final Report, Vol. 1; Theories, Concepts, 
and Ideas; Vol. II; Contexts, Findings, and Ideas; Vol. Ill: Ideas on Methods; 
Vol. IV; Tools and Ideas for Practice). MIT, Sloan School. 
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Archival Publications; 



Arthur, M. B., D. J. Levlnson, E. Schein, and M. A. Shepard (1984). Working 
u tth Careers . Center for Research in Career Development, Columbia University 
School of Business, 198A. 

Bailyn, Lotte (1985). Autonomy in the industrial R&D lab, Human Resource 
Management , 24, 129-146. 

Bailyn, Lotte. Issues of work and family in Organizations: Responding to 
social diversity. In Workinf^ with Careers , Artl r, et al., op.cit. 



A DEVELOPMENTAL THEORY OF WORK MOTIVATION 
NR 170-926 

Pennsylvania State University/Landy 

Scientific Objectives : Current motivational theories (e.g., reinforcement, 
need, social comparison, and cognitive) were examined with the intention of 
developing an integrated theory of motivation. The proposed theory was based 
on two assumptions: that individuals vary with respect to the mechanisms by 
which they are motivated; and that these mechanisms change over time. 

Approach : Comparisons were made of various incentives for individuals at 
different career stages. A sample of individuals was followed over a three^ 
year period to assess changes in the values they apply to different types of 
incentives. Questionnaires included biographical data, occupational/student/ 
military status, vocational preference, motivational measures, and self- 
ratings of self-esteem. 

Results and Conclusions : Traditional approaches to understanding work 
motivation have not been successful and so middle-range theories are necessary 
for progress in this area. Cognitive processes will play a major role in 
motivation theories of the future and various middle-range theories are likely 
to be more suitable for understanding some dependent variables than others. 
To facilitate such future research, a taxonomy of motivation theories was 
developed and examples of each category were analyzed. 

Potential Applications : This rese^>rch will help direct researchers to the new 
conceptions necessary in order to make motivational theories u. ^ful in 
predicting individual work performance. 

Reports ; 

Landy, Frank J. and Becker, Wendy S., (1985). Final Report : Adaptive 
Motivation Theory (ON^.-85-l). Pennsylvania State University, Paychology 
Department. 



ERIC 



61 



A TRIPARTITE APPROACH TO EVALUATING PERSONNEL /HUMAN RESOURCE DEPARTMENT 

EFFECTIVENESS 

NR 170-963 

Duke University/Tsui 



Scientific Oblectlves; The effectiveness of the personnel subunlt can be 
evaluated by the activities it performs, the criteria it optimizes, and the 
multiple constituencies it satisfies. This research tested several hypotheses 
specifying the -elationships among the personrel department's activities, 
criteria, reputation and the effectiveness of the larger organization in' 
which the subunit is located. 

Approach; A combination of interviewing. Delphi, and mailed surveys was used 
to collect data. Delphi was used to define the domain of personnel activities 
and evaluation criteria. Interviewing and surveys were used to obtain 
preferences for, and evaluation of, personnel department tasks by the multiple 
constituencies. The sample for the first study was 150 fie.ld personnel units 
in three muiti-divisionai corporations or institutions. The sample for a 
second study was 300 small-sized to medium-sized firms. Data included both 
subjective opinions and institutional records. Multivariate statistics were 
used to analyze the data. 



Results at. d Conclusions ; The research reveals that the personnel subunit 
performs nine general human resource functions. The nine functions include 
both routine, administrative tasks and proactive, stiategic activities. 
Clients placed high importance on both administrative services and employee 
relations activities. Actual personnel unit actions also emphasized 
consistency in policy applications. Effective personnel units were able to 
obtain support from their clients who have resources j-ich as time, ideas, and 
money, and provided solutions to critical organizationrl problems such as 
productivity and employee relations. Further, effective personnel units gave 
timely responses to client needs. Finally, the contribution of personnel unit 
activities to organizational effectiveness was observed by the significant 
correlations between this unit's innovativeness. efficiency and resource 
acquisition and organizational effectiveness criteria such as job 
satisfaction, organizational commitment, intention to stay with company, 
absenteeism, and perceived operational effectiveness. 

Pote.>tial Applications; Findings should lead to improved organization of 
personnel departments; definition of activities and evaluation criteria; 
allocation of resources and effort across multiple activities, criteria and 
constituencies; and an evaluation process that addresses the interests and 
requirements of the beneficiaries of personnel department efforts. 
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Reports ; 



Tsui. A. S. and Milkovich, G. T. (1985). Dimensions personnel department 
activities; an empirical study. (Res. Rep. ONR-85-i; . Duke University, 
Fuqua School of Business 

WORK AND THE PERCEIVED QUALi: / OF LIFE 
NR 170-964 

State University of New York, Buffalo/Rice 

Scientific Objectivec; The perceived quality of life (pQL) concerns how 
people feel about their lives, as a whole (overall pQL) or with regard to 
specific domains of life e.g., the perceived quality of work life). This 
research tested hypotheses concerning determinants of pQL and the manner by 
which work experiences affect overall pQL. 

Approach : Correlational analyses of questionnaire data were used to test 
these hypotheses. Such analyses were performed with large (N=1000) 
national survey data fcts obtained through ICPSR and with small (N's= 48-141) 
original data sets obtained from local surveys of employees, students, and the 
unemployed. Self reports of satisfaction, happiness, and related evaluations 
of life were the principal criteria in these analyses. Predictor variables 
included: a) job characteristics; b) importance of work and nonwork roles; 
c) conflict between work and nonwork roles; and d) three variables concerning 
outcomes associated with participation in different life domains (amount of 
each outcome received, amount wanted, and importance of the outcome). 

Results and Conclusions : 

Results generally supported hypotheses derived from the conceptual model of 
pQL. Specifically, (1) The sum of domain pQL scores (e.g., satisfaction with 
job or family) was predictive of overall pQL; (2) Work-family and work-leisure 
conflict were the only work-related variables with relationships to overall 
pQL that were mediated through causal pathways involving both the quality of 
work life and the qualify of nonwork life; (3) Conflict between work and 
family was highest for people ascribing high importance to both domains; (4) 
pQL vras better predicted from the discrepancy between what people get 
(outcomes) and what they want (standards) than from just what they get; (5) 
The utility of importance (values) in predicting pQL depended on the type of 
predictor scores with which importance information was used—when used with 
outcomes, importance scores enhanced prediction of pQL, buc when use^. with 
satisfaction score predictors, importance did not enhance prediction of pQL. 

Potential Applications : Prior research has shown that pQL is related 

to outcomes f both economic and humanistic concern to organizational decision 

makers, e.g., turnover, absenteeism, and physical health. Programs seeking to 

control such outcomes would be^^efit from a conceptual model clarifying the 

basic processes determining pQL, and the links between work experiences and 

pQL. 
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Reports; 



Rice, R. W., Bonacci.M. & Bunker. B. (1985). Involvement with Prior Job. 

Social Support, and the Perceived Quality of Life Among the Unemployed 

(Tech. Rep. NO ONR-3). (AD-A162102) . ^ — 

Rice, X.. Frone, M. & McFarlin, B. (1985). Inter-domain Fit and the Perceived 
quality of life (Tech. Rep. uNR-4) . (AD A161993)~! 

Rice. R., McFarlin. B.. Hunt. R. & Near. J. (1985). Job importanc e as a 
moderator of the relati onship between job satisfaction and life satlif^ion 
(Tech Rep. ONR-5). (AD-A162105) . 

Archival Publications ; 

Rice. R. W.. McFarlin. D. B.. Hunt. R. G.. & Near. J. P. (1985). Organizational 
work and the perceived quality of life; Toward a conceptual model. Academy of 
Management Review . 1985. . 296-310. (AD A161993). 
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H. PRODUCTIVITY IN ORGANIZATIONS 

In light of the changed focus of this Program in fiscal year 1985, we will 
not support new research on productivity. 
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THE DESIGN AND MANAGEMENT OF WORK TEAMS 
NR 170-912 

Yale University/Hackman 



Sclentlf .c Object Ives; To advance theory about small group performance; 
to generate findings that are of practical use in the design and management of 
teams in organizations, and to develop research methodologies that are 
particularly well-suited for the study of group effectiveness. 

Approach; Intensive study of a variety of work teams in organizations, using 
multiple methodologies (including self-report, observational, interview, and 
archival methods). The main sample is about 30 highly diverse teams which 
have been studied in depth. In addition, a large number of teams, studied 
less intensively, are being used for psycho-netric analyses aimed at refining 
the data-collection instruments. Finally, a subset of teams in the main 
sample are being used in action-research projects aimed at assessing the 
usefulness of the concepts and methods in bringing about improvements in 
actual team performance. 

Progress; Data collection is complete and data analysis is nearing completion. 
Findings to date suggest that the design of work groups (i.e., their 
composition, their tasks, and the core norms that guide member behavior) and 
the properties of the organizational context in which they operate are 
especially potent in affecting team performance. These findings provide a 
way of understanding group effectiveness that contrasts with previous research 
focused on the patterns of interpersonal behavior that occur among group members. 
Work is underway on a number of major publications. 

Potential Applications; The normative model of team effectiveness being 
tested and refined in the research program can be used to guide the design 
and management of groups that perform tasks in organizational settings. 
Specific strategies for using the conceptual model and research findings for 
this purpose are presently being developed and assessed. 

Archival Publications ; 

Hackman, J. R. (1985). Doing research that makes a difference. In E. E. 
Lawler, A. M. Mohrman, S. A. Mohrman, G. E. Ledford & T. G. Cummings (Eds.), 
Doing rese arch that is useiul for theory and practice . San Francisco; 
Jossey-Bass. 

Hackman, J. R. (1985). The commitment model; From "whether" to "how." In 
K. B. Clark, R. H. Hayes & C. Lorenz (eds.). The uneasy alliance; Managing 
the productivity-tech nology dilemma. Boston; Harvard Business School Press. 

Hackman, J. R. (1986). The design of work teams. In J. Lorsch (Ed.), 
Handbook of organizat ional behavior . Englewood Cliffs, NJ; Prentice-Hall. 
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INFORMATION PROCESSING 
NR 170-950 

Texas A&M University /Daft & Griffin 



Scientific Objectives ; This effort aims to develop new and more 
sophisticated theoretical frameworks for understanding different methods of 
information-seeking and information dissemination by managers as a function 
of variables such as the complexity of the tasks they deal with, the 
manager's level in the hierarchy, and changes in the organization's 
environment • 

Approach ; Theory and model building are undertaken to explicate how 
information processing is likely to occur in organizations. Laboratory 
studies are conducted to test how various kinds of information-processing 
manipulations influence perceptions of the work place. Field research using 
survey instruments, archival documents, and interviews provides data to 
refine the models and extend the laboratory findings. 

Progress ; Work is proceeding on three tracks. First, a model has been 
proposed and tested, dealing v ith the relationship between media richness and 
managerial information procesi;ing. Findings indicate that managers tend to 
prefer rich, oral, face-to-face media, when learning requirements are high, 
but written media when learning requirements are low and routine. Second, a 
model has been developed and partially tested, with results suggesting that, 
with regard to an employee's perception of the work environment, information 
received from other people is as important as are the objective facts of the 
situation. Third, a model is under development dealing with organizations 
viewed as systems for interpreting information; testing will follow 
development. 

Potential Applications : Results will help to define the most effective 
managerial behavior in a variety of situations. The work on matching the 
choice of information medium to the nature of the information to be 
communicated will enhance the design of management information systems for 
maximal utilization, while work on managers' choice of cues and media for 
transmitting information will contribute to the training of managers for 
maximal effectiveness. 

Reports ; 

Griffin, R., Bateman, T., Wayne, S., and Head, T. (198^). Objective and 
Social Factors as Determinants of Task Per ceptions and Responses: An 
Integrative Framework and Empirical Investigation (TR-ONR-DG-09) . Texas 
A&M University, Department of Management. 

Daft, R. and Lengel, R. (1984). A Proposed Integration Among Or ganizational 
I nform at ion Requirements, Media Richness and Structural Design (TR-ONR-DG- 10) 
Texas i^6M University, Department of Management. 

Giroux, G.A., Mayper, A. G., and Daft, R. L. (1984). Toward a Strategic 
Contingencies Model of Budget Related Influence in Municipal Government 
O rganizat ions (TR-ONR-DG-1 1 ) . Texas A&M University, Department of Management . 
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Macintosh, N. B. and Daft, R. L. (1985). Technology. Personal Attributes and 

the Perceived Amount an d Focus of AccountlTrrTnd Information Sy stem T)^ 

(TR-ONR-DG-12). Texas A&M University, Department of Management. 

Macintosh, N. B. and Daft, R. L. (1985). Management Control Systems and 

Interdepen dencies; An Empirical Study (xR-ONR-DG-13) . Texas A&M 

University, Department of Management. 

Head, T. C. , Yates, V. L. Griffin, R. W. and Bateman, T. S. (1985). The 

Priming E ffect In Task Design Research (TR-ONR-DG-14) . Texas A&M 

University, Department of Management. 

Griffin, R. w.. Sklvlngton, K. D., and Moorhead. G. (1985). Symbol ic and 

Interactional Perspec tives on Leadership; An Integrative Fra^i^ ^ 

(TR-ONR-DG-15). Texas A&M University, Department of Management. 

Griffin, R W., Bateman, T. S. (1985). Job Satisfaction and Or ganizational 
Commitment (TR-ONR-DG-16) . Texas A&M University, D epartment of Management. 



PERFORMANCE APPRAISAL ~ MEMORY AND RECALL PROCESSES AS THEY AFFECT APPRAISFR 

ACCURACY 

NR 170-960 

University of Connectlcut/Barnes-Farrell 



Scientific Objectives; The research is designed to explore performance 
appraisal from a process perspective. Specifically, it focuses on gaining a 
better understanding of the ways in which memory and recall processes affect 
appraiser accuracy. 

Approach: Two phases are Involved; (1) the development of stimulus 
materials and measures of accuracy in performance appraisal; (2) the use of 
these materials in laboratory studies to test hypotheses generated from social 
cognition models regarding the ways in which memory and recall processes 
affect appraiser accuracy. 

Progress; Completed studies include one examining the effect of 
task salience on appraisal accuracy and behavior recall, and examining the 
process underlying occupational age-typing and the nature of employee age 
stereotypes. Additional studies which are currently in progress include an 
examination of the effects of commitment to an evaluation on the evaluation of 
new performance information, and a continuing investigation of the processes 
leading to occupational age-typing. 



Potential Applications; (1) the development of a delivery system for 
performance-appraisal training based on the interactive video svstem- 
set of recoiranendatlons for improving real-life performance appraisals. 
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DECISION MAKING IN DISPERSED, COMPUTER-INTERACTIVE GROUPS 
NR 170-965 

University of Arizona/Connolly 



Scientific Objectives ; Investigate the communication networks that 
develop among multiple decision makers interacting over simple computer-based 
information channels. It is assumed that network structure will, to an 
important extent, emerge from user choices, perhaps maladaptively for the 
decision process overall. 

Approach : The project is primarily laboratory-based, empirical research 
using student participants solving simulated business or other problems. 
Participants operate interactively on networked microcomputers with on-line 
data acquisition. The major focus is on the patterns of information exchange 
that emerge among participants as a function of task demands for coordination, 
the incentive scheme operating on information suppliers, the cost and quality 
of the information available, and the size of group performing the task. 

Progress ; Normative, cost-benefit factors appear to shape acquisition only 
weakly, and large departures from network optima are typical. The PI is 
currently exploring the effects of task structure, cueing from other 
participants, and direct incentives on individual acquisition behavior, with 
a view to establishing conditions under which emergent networks will develop 
towards or away from optimal. 

Potential Applications : Military operations frequently require decisions to 
be made by a group of individuals out of range of face-to-face interaction 
with one another/^ Coimnand and control systems are designed to support rapid 
decision making under such conditions. Designers of these systems need to 
understand the cognitive and interactive preferences that users bring to such 
networks, since design characteristics can influence decision quality. 
Furthermore, for reasons either of system malfunction or the need to respond 
to unique conditions, sometimes users will override a designed system and 
create an emergent network structure. There is reason to believe that 
emergent networks may often contain biases which lead to serious mis-matches 
with the demands of given decision tasks, and a better understanding of these 
processes could inform the design of future command and control systems. 

Reports : 

Connolly, T. & Thorn, B. K. (1985): Predecisional information acquisition : 
Effects of task variables on suboptimal search strategies . (Technical Report 
//ONR 85-1). Tucson: University of Arizona, August. 



THE MINNESOTA INNOVATION RESEARCH PROGRAM 
NR 170-966 

University of Minnesota/Van de Ven 

Scientific Objectives : This research program includes eleven longitudinal 
studies on the management of a wide variety of technical and administrative 
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innovations. The common research questions being examined in all the studies 
are: How do innovations develop over time? What organizational arrangements 
facilitate and inhibit innovation? How do environmental, organizational, 
group, and individual factors influence innovation success over time? 

Approach ; The interdisciplinary research group is undertaking a longitudinal 
study of a set of major innovations, including: Development of galium arsenide 
integrated circuits, industrial and surgical product innovations, development 
of complex Naval systems, development of government strategic plai -ing systems, 
development of vertically-integrated hospital systems, and human resources 
management innovations. The common research framework being used in all the 
studies posits that the process of innovation is essentially the evolution of 
innovative ideas which are carried out by people who are engaged in transactions 
with others within an institutional setting. Significant changes in these factors 
represent an event. A careful mapping of events over time is the central common 
task for all studies in the innovation research program. 

Progress : Case histories have been generated for each innovation from its 
inception to the present, and baseline data have been collected regarding 
their environmental and organizational settings. 

Potential Applications : The longitudinal study of the developme : of a naval 
weapons system by Navy contractors will have direct and immediate 
applications to the Navy. Combined with the other studies, it will contribute 
to identifying general principles for the management of innovation. 



Reports : 

Van de Ven, A. H. Leadership and organization culture: navigating the 
tricky currents (ONR-TR-2). University of Minnesota, The Strategic Management 
Center, Nov 1984. 

Van de Ven, A. H. An examination of alternative forms of fit in contingency 
theory (ONR-TR-3). Univeristy of Minnesota, The Strategic Management Center, 
Nov 1984. 

Van de Ven, A. H. Dynamics of interorganizational coordination (ONR-TR-4) . 
University of Minnesota, The Strategic Management Center, Nov 1984. 

Van de Ven, A. H. The concept of fit in contingency theory (ONR-TR-5). 
University of Minnesota, The Strategic Management Research Center, Nov 
1984. 

Van de Ven, A. H. Integrating human resource management and corporate strategy; 
A preview of the 3M Story (ONP-TR-6). University of Minnesota. The Strategic 
Management Research Center, Nov 1984. 

Van de Ven, A. H. Central problems in the mangement of innovation (ONR-TR-8). 
University of Minnesota, The Strategic Management Research Center, Nov 1984. 
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Gresov, R. and Van de Ven, A. Testing the Organizational Assessment 
Model of Work Unit Design; A Systems Approach (ONR-TR-8). University of 
Minnesota, The Strategic Management Research Center, Nov 1984. 



LEADERSHIP: THEORY AND APPLICATION TO CRISIS MANAGEMENT 
NR 170-970 

Harvard Unlversity/Heifetz 

Scientific Objectives* The ultimate aim Is the specification of a quantitative 
theory of group processes involved in leadership phenomena. Intermediate 
goals are continued expansion of a new set of theoretical principles, and a 
test of how well these principles are alle to model events that occurred 
during an actual national security crisis. 

Approach: Theoretical work will be reflected in a book manuscript. Crisis 
modeling will be based on analyses of tapes and transcripts (recently 
released) concerning the Cuban missile crisis. The analysis will focus on one 
of the central propositions In the theory, viz., that the Importance of Issues 
being worked on correlates directly with the strength of avoidance behaviors 
shown by decision-making participants. 

Progress: Work began In late September, 1985 and no Status Reports have yet 
been received. 

Potential Applications: The theory being developed Is designed to be 
especially useful in understanding the leadership of decision-making groups 
operating under high stress. Effective Navy leadership under such conditions 
Is crucial to Navy's operational readiness. 

THE FOLLOWING UNITS WERE COMPLETED DURING THE LAST YEAR 
****** 



STRATEGIC MANAGMENT FOR ORGANIZATIONAL EFFECTIVENESS: THE IMPACT OF 
HUMAN RESOURCE PLANNING ON RETENTION AND RELATED ISSUES 
NR 170-949 

University of Pennsylvanla/Oppenhelm 

Scientific Objectives: The study explored the relationships among a number 
of human resource variables which bear upon retention of an appropriate work 
force. These factors (planning, Incentives, mobility and decisions to stay 
with or leave the organization) were believed to Influence the effectiveness 
of an organization. 

Approach: The research took a multl-methodologlcal , Interdisciplinary approach 
to the research problem described above. Specifically, comparative case 
studies of "exemplar" organizations were used to Identify Important 
relationships among the variables listed above. In addition, perceptions of 1) 
planning and Incentive systems, 2) the effect of expectations of mobility on 
the boss/subordinate Interaction and developmental activities, and 3) the 
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effect of organizational signals on the decision to stay with or leave the 
organization were assessed. Approximately one hundred middle-level personnel 
from each of six sites were used as respondents. Three instruments were used 
to collect data: 1) open ended interviews, 2) questionnaire and structured 
interviews, and 3) scenarios and structured interviews. 

Potential Applications; Specific findings can assist human resource policy 
makers and managers by influencing the incentives, development activities, and 
communications used to retain an appropriate work force. The Navy can 
benefit by learning which of its practices correspond to those of the civilian 
exemplar institutions. This comparison is made easier because a Navy group 
was used as a research site; differences between the Navy and other sites are 
thereby elucidated. 

Results and Conclusions; The key findings were: 1) a model of job movement 
which took expectations into account was a good predictor of the way in which 
a manager and his boss divided tasks; 2) the rate of movement from one job to 
the next was more rapid in organizations where jobs were clearly and narrowly 
defined than when jobs evolved and expanded over time; 3) signals from the 
organization which were public, positive and relevant to a manager's goals 
increase the likelihood of his remaining with the organization; 4) managers 
were more likely to see themselves as resource constrained if goal setting 
and resource allocation were decided at different levels in the organization; 
5) informal incentives were more salient to middle managers than most formal 
incentives. 

The results indicated that the strategic considerations of these organizations 
were well served by their human resource practices although the configuration 
of policies and practices differed. The research also indicated that these 
effective organizations exhibited greater flexibility in interpreting policy 
and enforcing boundaries in areas in which they needed information to 
innovate and adapt. 

Reports : 

Oppenheim, L. (1985). Strategic management for organizational effectiveness: 
The effect of human resource planning on retention and related issues (Final 
Report). University of Pennsylvania, The Wharton School, Dec 1984. 
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INCREASING PRODUCTIVITY THROUGH SOCIAL STRUCTURE 
NR 170-954 

University of North Carolina/Latane 



Scientific Objectives : To determine whether negative effects on productivity 
(termed "social loafing") can be eliminated and reversed by changes in group 
structure and organization. 

Approach ; The research consisted of experiments designed to explore the 
psychological processes underlying social loafing, to discover on what kinds 
of tasks and in what kinds of job settings it occurs, to find out how it 
relates to job satisfaction and the quality of work life, and to determine 
whether the forces leading to social loafing are moderated by cultural back- 
ground, philosophical or political ideology or personal li:"e history. The 
research was conducted in the context of a custom-designed ^ computer-based, 
telecommunicating ongoing work organization of 100 military-age part-time 
employees; 20-year olds were hired to work several hours a week for a year In 
front of computer terminals, communicating with each other and with their 
supervisors only by machine. 

Results and Conclusions : The data indicate that effective i;roui> communication 
can take place in the computer-mediated environment and that anonymity had little 
effect on either performance quality of patterns of influence, perhaps 
because of the relative weakness of the identified condition in this 
research. The data provide evidence for the feasibility of Informal interaction 
and substantial communication of socio-emotional content via computer networks. 
The data suggest that the computer may not be as poor a medium for this sort 
of interaction as once supposed. Not the least of the accomplishments has 
been the development of an ongoing data-collection facility vhich promises to 
be a major resource for scientists doing research on individuals working in 
groups and /or by themselves. 



Reports : 

Barefoot, John, Wiggins, Beverly and Latane, Bibb (1985). Computer- 
Administered Panel Study: Methodology (Technical Report). Uriversity of 
North Carolina, Poychology Department. 

Barefoot, John, Wiggins, Beverly and Latane, Bibb (1985). Compter-Mediated 
Communication: Decision-making and Informal Interaction (Technical Report). 
University of North Carolina, Psychology Department, Feb 1985. 

Latane, Bibb (1985). Increasing Productivity through Social Structure: 
Final Technical Report . University of North Carolina, Psychology Department, 
Feb 1985. 
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MANAGEMENT AND WORK FACILITATION: AN APPROACH TO PRODUCTIVITY 
NR 170-958 

University of Maryland /Schneider & Schooman 

Scientific Oblectives ; Conceptualization and measurement of work 
fncilitation and management behavior, and establishment of the ways 
management facilitates and inhibits employee performance. 

Approach; The work focused on work groups, not individuals. The goals were 
the development of generic procedures for diagnosing the presence of unit-level 
facilitators (and the absence of barriers) a^ well as management behaviors 
(e.g., planning, organizing, delegating) that are related to them. The 
approach began with interviews of work units, moved to development of 
unit diagnostic-checklists, to administration of the diagnostic 
checklists to units in a group session. Criterion data for three samples 
of work units were obtained to allow for inferences about the validity of 
the diagnostic procedures. 

Results and Conclusions; In three different organizations, three versions 
of a Work Facilitation Diagnostic (WFD) were developed as a basis for differ- 
entiating unit functioning within the organizations. Katz and Kahn's 
conceptualization of subsystem functioning served as the framework for WFD 
development. In each organization, each work unit completed a WFD and the head 
of each unit completed a job-analysis survey describing his or her management 
activities. Criterion data were available on every unit studied (teaching 
ratings for university departments, sales for telemarketing sales units, 
and unit ratings of effectiveness for the DcD sample). In all studies some 
significant relationships were observed between unit WFD data and unit 
effectiveness suggesting the validity of the procedure used for the 
development of WFDs. 

Potential Applications ; The WFD itself is partially, not totally, generic, 
but the framework and methodology for developing WFDs is totally generic. 
As such, it is useful for diagnosing a broad range of work units in a 
variety of organizations. A unique fe ture of the WFD is its focus on 
organizational functioning in all of the important subsystems of 
organizations. A study is now underway, with 6.2 funding, to apply these 
ideas to a Navy organizational unit (see Cluster I, NR A75-OA2). 

Ar chival Publications ; 

Schneider, B. (1985). Organizational uehavior. Annual Review of 
Psychology , ^6, 573-611. 

Schneider, B., & Bowen, D.E. (1985). Employee and customer perceptions of 
service in banks: Replication and extension. Journal of Applied Psycholoev, 
70, A23-A33. ^ ^ ^ 

Schoorman, F.D., & Schneider, B. (1986). Grappling with work facilitation. 
In F.D. Schoorman and B. Schneider (Eds.), Work facilitation and 
organizational effectiveness . Lexington, MA: Lexington Books. 
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Moeller, A., Schneider, B. , & Berney, E. (1986). Development, reliability, 
and validity of the Work Facilitation Diagnostic; Operationalization of the 
Katz ard Kahn subsystem model of organizational effectiveness. In F.D. 
Schoorman end B. Schneider (Eds.), Work facilitation and organizational 
effectiveness. Lexington, MA: Lexington Books* 



PERFORMANCE APPRAISAL: A PROCESS APPROACH 
NR 170-961 

Purdue University/Ilgen 



Scientific Objectives : The objective was to better understand the judgment 
pro:ess by which individuals judge, as accurately as possible, the work 
periormance of others. This judgment practice is very complex psycho- 
logically requiring raters to perceive, store, and recall relevant information 
about others' performance on the job. 

Approach ; Laboratory and field research investigated the variables that 

influence accurate judgments. Laboratory research frequently used simulations 

in which participants role-play supervisors with many tasks, one of 

which is to conduct performance appraisals. Field research required employees, 

who do appraisals as part of their job, to view tapes so that accurate raters 

can be compared to inaccurate ones In order to discover individual and situational 

differences between the two groups. 

Results and Conclusions : The series of research studies showed that 
cognitive perforiiiance-appraisal accuracy was influenced by the nature of the 
performance uf ratees, and more importantly, that cognitive categories 
influenced performance accuracy. In the latter case, research with nurses 
developed ways to assess the cognitive categories used by the nu'^S'^s and to 
assess the effects of various characteristic ' of these categories on 
ratings. Finally, training programs designed to influence appraisal accuracy 
were found to do so. This work implied that cognitive-process approaches to 
performance appraisal are useful for explaining some of the factors affecting 
rating accuracy. However, the amount of variance these factors control in 
actual appraisal settings may be more limited than initially estimated. 

Potential Applications : Results form the basis for improved design of 
performance appraisal systems and development of training programs for using 
these systems. Past research hac focused almos^ exclusi^'^'ily on the design of 
rating scales and training to eliminate rating errors. Our research suggests 
expansions of approaches to incornorate the way in which raters make inter- 
personal judgments. 

Reports : 

Ilgen, D. R. (1985). Laboratory research! A quesf^on of when, not if 

(Techn-^-^al Report No. 85-1). Michigan State University, Departments of 
Psychology and Maragement, March 1985. 
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Kozlowski, S. W. ... Kirsch, M. P. , & Chao, G. T. (1985). Job knowledge, 
^_atee famtlarlty. conce ptual similarity, and halo error; An ex nloraf^"n 

(Technical Report No. 85-2) Michigan State University. Departments of 

Psychology and Management. 

Ilg«.n, D. R., k Wiggins, A. (1985). The passage of time; A neglec ted 
factor in the KOai s^.tt lng-to-perf ormance to-feedback sequence (Ttrhnir^^ 
Report No. 85-3) Michigan State University, Departments of Psychology and 
Management, August 1985. e.7 " 

Archival Publications ; 

Ilgen, D. R., & Favero, J. L. (1985). Methodological limitations of social 
psychological literatures for the understanding of performance appraisal 
processes. Academy of Management Review, 10, 311-321. 
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I. SPECIAL PROJECT: MANPOWER R&D PROG^ • - 



This is an interdisciplinary program of exploratory development managed 
by Dr. Stan Collyer in the CNR Technology Programs Directorate. 
Scientific Officers for these contracts are located in the Psychological 
Sciences Division and the Mathematics Program of the Office of Naval 
Research. This brochure includes descriptions only for those contracts 
monitored by a Scientific Officer in the Psychological Sciences Division. 

The program was initiated as a direct result of the national policy that 
replaced the draft with the all-volunteer force as the source of defense 
manpower. The ONR manpower program has dealt with a wide range of programs, 
both in response to. and in anticipation of. expressed Navy needs. A major 
aim has been the development of techniques and models to be used by Navy and 
Marine Corps managers in dealing with manpower and personnel issues. 
Research is conducted in universities and other institutions; methods are 
varied and include experimentation, statistical analysis, simulation, and 
modeling. This program is closely coupled with the operating arms of the 
Navy and Marine Corps through the mechanism of a planning committee, whose 
members include ONR Scientific Officers, the Naval Civilian Personnel Command, 
the Naval Military Personnel Command, the Navy Recruiting Command, the 
Navy Personnel Research and Development Center, several directorates in the 
Office of the Chief of Naval Operations, and the Navy Secretariat. 

During the year reported on herein, the research focused on development of 
reenllstment decision aids, factors that facilitate work performance, and new 
concepts and techniques for assessing military personnel effectiveness. 
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MANPOWER RESEARCH AND ADVISORY SERVICES 
NR 170-032 

Smithsonian Institution/Sinaiko 



Scientific Oblectxves ; The contract s-.rves as a bridging mechanism for naval 
manpower issues and the research t.at addresses them. Substantive areas 
include: the retention of skilled personnel, the supply of manpower to the 
Navy, personnel turnover, and the reserve force. The contract also arranges 
for the services of expert consultants in the behavioral sciences. 

^E2I2ach: Provides for information exchange through a navywide committee. 

ner^i^fl^'^T/?.'? ^f"/"" "r"'^'" °""P^tional correlates of 

personnel attrition) and conducts statistical analyses on them. Chairs 
an active international collaboration among military manpower researchers 
in the U. S.. Canada, Australia, New Zealand, and the United Kingdom. 

Progress ; Special symposia and meetings have been planned and conducted on 
such subject's as personnel supply forecasting, recruiting advertising 
effectiveness , and innovations in survey research methodology. In 1985. for 
example, a workshop on Hispanic personnel was organized. 

Potential Applications; Information about new research and about innovative 
manpower practices and policies, acquired through inter-service (U.S.) and 
international collaboration and exchange, reaches the U.S. Navy through the 
Manpower R&D Planning Committee. Subjects include but arc not li^iited to 
career information, retraining, expanding the supply of manpower, and new 
ability measures, both cognitive and non-cognitive. 



Reports ; 

Sinaiko. H. W. . Curran, P. M. King. B. T.. & Schneider. J. M. (1985). 
Hispanic Subpopulat lons and naval service (Tech. Rep. SI/MRAS/TR-1 1) 
Smithsonian Institution. (AD-A154662) . 

Sinaiko, H. W. (1985). Part-time soldiers, sailors, and airmen; Reserve 

force manpower in Austr alia. Canada. New Zealand, the UK. and the U.S. 

(Tech. Rep.). The Technical Cooperation Program. ' 

Archival Publications ; 

Sinaiko. H. W. (1985). European psychology and the Office of Naval Research. 
American Psychologist. hO , 861-862. 
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THE DRILL INSTRUCTOR: STRESS, COPING SKILLS. AND CAREER DEVELOPMENT 
NR 475-009 

University of Washington/Sarason 

Scientif ic Objectives; To develop and evaluate a unified program dealing 
with stress-coping skills for both the Marine Corp Recruit Depot, San Diego, 
and the Marine Corp Recruit Depot, Parris Island. The aim of the project is to 
produce relevant televised modules and ancillary material, and to conduct an 
evaluation of all materials. 

Ap proach ; The project has several components. One of these is the production 
of televised modules and vignettes for use as part of fhe Drill Instructor 
School curriculum. Manuals for use by Drill Instructor School instructors 
and a study guide for Drill Instructor School students will be developed. All 
of these materials will be evaluated using objective and subjective indices. 

Progress ; The televised modules and vignettes and will be completed by the end 
of 1985. The program will then be evaluated. 

Potent ial Applications ; The materials developed should be of considerable 
value in strengthening the stress coping skills of drill instructors. This 
should result in improvements in the drill instructors' performance and the 
effectiveness of the recruits they train. 



Funding : USMC 

AN AUTOMATED AID FOR PERSONNEL RETENTION 
NR 475-014 

University of Washington/Beach & Christensen-Szalanski 

Sci entific Objectives ; (1) to develop a technique for identifying personnel 
who are up for re-enlistment who a) clearly are going to re-enlist, or b) who 
clearly are not going to re-enlist, or c) whr ire undecided, and (2) to 
develop a video/computer system for helping them think through their 
re-enlistment decision. The system serves to idenvify the three groups, 
and pinpoints for the third group those areas of concern that may be 
addressed in career counseling. The counseling consists of supplying 
relevant information in an attempt to aid in clarification of the decision. 

Approach : The system uses simplified multi-attribute utility theory (MAUT) 
for a hierarchically organized set of categories of considerations that have 
been obtained in face-to-face interviews with personnel from submarine, air, 
surface, and shore commands. The MAUT hierarchy is presented via computer 
and detailed instructions are keyed to the presentation using a laser-beam 
videodisc recording of the step-by-step procedure. 
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Progress; Interviews have been completed, the hierarchy has been 
constructed and has been reviewed by experienced Ni;val career counselors for 
relevance and authenticity, videorecordlng has begun, the computer /video 
hardware has been assembled, and programming is nearly completed is 
expected that the system will be ready for installation in a command for 
initial testing early in 1985. 

Potential Applications ; This system will enable the Navy to 
ascertain what factors are most important in reenlistment decisions, and 
will help potential reenlistors to more carefully think through their 
decisions. 

Reports ; 

Christensen-Szalanski. J. J., Northcraft. G. B. Patie nt Compliance Behavior; 

The Effects jf Time on Patients' Values of Treatment Regimens fTR-SS-n . 

University of Washington, Department of Psychology, March, 1985. 

Beach, L. R. , Weinstein, B. L. & Christensen-Szalanski, J. J. U.S. Navy 

Reenlistme nt Decision Aiding Scheme . (TR-85-2;. University of 

Washington, Department of Psychology, March, 1985. 



MARGINAL-MAXIMUM-LIKELIHOOD ESTIMATION IN ITEM-RESPONSE THEORY 
NR 475-018 

National Opinion Research Center/Bock 

Scientific Objectives; In this work, the usefulness of recent theoretical 
developments in marginal-maximum-likelihood methods for c ^timating item- 
response functions, for gauging the dimensionality of tests, for simultaneous 
multidimensional adaptive testing, and for estimate .g the trait distributions 
ot selected subpopulations were explored in a military context. 

Approach; limpirical studies employing Armed Services Vocational Aptitude 
Battery (ASVAB) data from the Profile of American Youth were used. The 
investigator developed a full-information factor analysis of the data and 
examined the performance of various indices of the number of factors 
Three-parameter logistic models were fit to all 10 ASVAB subtests and 
cross-validated. Estimated item-information functions were examined in 
light of ^tem content and evidence for multidimensionality . Several methods 
for estimating subtest trait distributions were also examined and compared. 

Results and Conclusions; (a) A four-pliase Bayes constrained estimation 
procedure has extended Bock's earlier work to the three-parameter-logistic 
situation. These procedures have been successfully applied to the Profile of 
American Youth data set. (b) Analysis of the dimensionality issue suggested 
that the Auto-Shop subtest is two-dimensional, (c) A multiple-category model 
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for incorporating information from wrong responses to multiple-choice items has 
been developed and tried out with the ASVAB data set. 

Po tential Applications ; (a) The multiple category model makes possible the 
recovery of information available in wrong responses to multiple-choice test 
items and can thus increase the speed and accuracy of computerized adaptive 
testing. The model also provides so-called category operating 
characteristics. These functions can be used by test designers to assess the 
effectiveness of distractors in multiple-choice test items, (b) The 
full-information factor analysis developed in this work ought to replace 
traditional linear factor analysis. The full-information approach not only 
conforms more closely to t:he latent-trait non-linear notion of dimensionality, 
but also reduces the likelihood of obtaining spurious guessing factors. 

Funding ; ONR Manpower R&D Program. 

Reports ; 

Bock, R. D., Gibbons, R., & Muraki, E (1985). Full-inform ation item factor 
analysis (Technical Report 85-1). Chicago, XL; National Opinion Research 
Center, University of Chicago. 

Mislevy, R. J. & Bock, R. D. (1984). Item operating charac teristics of the 
Armed Services Vocational Aptitude Battery (ASm)_, form 8A (Technical Report). 
Chicago, XL; National Opinion Research Center, University of Chicago, 

Thissen, D. & Steinberg, L. (1984). A response model for m ultiple choice 
Items (Technical Report). Chicago, XL; National Opinion Research Center, 
University of Chicago. 

Archival Publications ; 

Mislevy, R. J. vl984). Estimating latent distributions. Psychometrika , 49, 
359-381. 

Thissen, D. & Steinberg, L. (1984). A response model for multiple choice 
items. Psychometrika , 49 , 501-519. 



EVALUATION AND IMPROVEMENT OF TEAM EFFECTIVENESS IN ANTI-SUBMARINE WARFARE 
COMBAT EXERCISES 
NR 475-026 

Georgia Institute of Technology/james 



Scientific Objec ives ; (1) Develop empirically validated causal model of 
Anti-Submarine Warfare (ASW) team effectiveness; (2) develop empirically 
validated causal models of the processes through which the effectiveness of 
teams determines the performance of multi-team "Units" of ships and aircraft 
operating in coordinated combat exercises. 
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Approach ; Data on unit effectiveness, team effectiveness, and causes of 
performance effectiveness are being collected at the ASW Training Center 
Pacific during simula^ed scenarios in which the Unit's task is to locate 'and 
asstroy enemy submarines. Primary sources of data are effectiveness 
evaluations furnished by ASW instructors through behavioral ratings and 
reports on causes of team effectiveness (e.g., communication, manning/ 
staffing motivation, leadership, stress, tean cohesiveness and morale) from 
team members and ASW instructors via questionnaiies. Empirically validated 
models are being constructed using confirmatory-analytic and time-series- 
analytic procedures. 

Progress; Progress includes the development and administration of team 

readiness questionnaires, the construction of the criterion rating form now 

being completed by ASW instructors for each graded exericse, the formulation 

of additional instruments to assess psychological variables not addressed in 

the rating form, the creation of a macro, unit-level measure of performance, 

and the development of causal models which hypothesize the determinants of 

team performance and, perhaps more importantly, specifv the causes of overall unit 

performance in terms of particular aspects of individual team effectiveness. 

Of special interest in empirically validating these models is the new 

criterion rating form used by the ASW instructor to rate team performance 

Based on a highly detailed, behaviorally-oriented , contact-bv-contact task 

analysis, this criterion rating form serves as the final grade sheet for ASW 

teams. In addition, it serves as a measure of the team-level process 

variables which contribute to unit level effectiveness. 

Potential Applications; The empirically validated causal models of team and 
unit effectiveness generated through this research will be transl?^ed into 
useful products for ASW training by (a) developing training modules for ASW 
tactical decision-making, (b) contributing to the design of computer 
simulations for shipboard and off-site, shore-based ASW team training, and 
(c) constructing indicators for assessing team readiness. 

Reports ; 

James, L. R. . and Tetrick. L. E. (1984). Confirmatory analytic te sts of three 

£f - al models relating job perceptions to .job satisfaction (Tech. Report 

Gl v«R-6). Georgia Institute of Technology, Psychology Department, 
December 1985. 

James, L. R.. Demaree, R. and Mulaik. (1985). S. A. A note on validity 
generalization procedures (Tech. Report GT-ONR-7) . Georga Institute of 
Technology, Psychology Department, January 1985. 



Archival Publications 



i'.Ti: l^^^^^^^- L. E. (1986). confirmatory analytic tests of three causal 

^l! 77-82 perceptions to job satisfaction. Jou rnal of Applied Psychology 
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PREDICTING MILITARY PERFORMANCE FROM EDUCATIONAL AND BIOGRAPHICAL INFORMATION 
NR i^lb-Qn 

Human Resources Research Organization/Reigelhaupt 

Scientific Objectives ; In previous work the investigators developed a survey 
instrument known as the Educational and Biographical Information Survey 
(EBIS) and administered it to large numbers of military applicants and 
recruits. This project is cross-validating EBIS items previously found to be 
the best predictors of 6-month attrition with attrition experience at 12 & 24 
months, with pay-grade data at 30 months and with reenlistment 
recommendations/decisions . 

Approach : By linking EBIS response data to Defense Manpower Data Center 
(DMDC) records, investigators are identifying EBIS respondents' attrition 
status at 6, 12, & 24 months, their pay grade at 20 months, and reenlistment 
recommendations/decisions. Covariance matrices for predictor and criterion 
variables are being examined and discriminant/regression analyses are being 
performed to identify the types of biographical information which 
significantly improve predictions of military performance. All analyses are 
being made both for the total DoD accessions and for individual Service 
accessions. 

Progress : A detailed comparison of data on the performance of EBIS and a 
variety of Service and DoD biographical inventories have been completed. 
Analyses indicated that items dealing with school behavior and attitudes are 
most consistently effective. In other work the predictive ability of various 
indices of past behavioral difficulties was compared: Defense Central Index 
of Investigation (DCII) status, moral waiver status, EBIS self report 
arrest scale score, and EBIS self report drug/alcohol scale score. Of these 
all except moral waiver status were found to be statistically related to 
attrition during the first 12 months of service. 

Potential Applications : This project is part of a multiple agency effort 
aimed at developing a pool of validated items from which the Services can 
draw questions for military candidate screening instruments to supplement 
cognitive measures such as ASVAB. 

Reports : 

Laurence, J.H. & Means, B. (1985). A description and comparison of 
biographical inventories for military select Ion (Teciinical Report 
FR-PRD-85-5) . Alexandria, VA: Human Resources Research Organization. 



EFFECTS OF JOINT SERVICE ADVERTISING ON RECRUITMENT OF MILITARY PERSONNEL 
NR 475-029 

University of Pennsylvania/Carroll 



Sc ientific Objectives : Assess the hypothesis that the same recruitment 
effectiveness can be achieved by increasing the Joint Service Advertising 
budget significantly while decreasing the overall DoD Active Enlisted NPS 
(Non-prior Service) Recruitment advertising budget. 
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AE£roach: An experimental design was developed for implementation in the 
field which would entail testing two advertising dimensions siEultaneously : 
The level of the total DoD Active Enlisted NPS recruitment advertising 
budget, including both Joint and Service-specific and the mix of joint and 
service-speci. .c advertising within each total advertising expenditure 
level. The field test currently underway includes combinations of total 
advertising budgets (i.e.. current, higher, and lower) and various 
proportions of that budget dedicated to Joint advertising (i.e. low 
current, and high) . ^ 

Progress: The complex and unique advertising data base which this effort set 
out to develop IS virtually complete. The criterion measures have shown both 
predictive validity and appropriate statistical sensitivity. 

Potential Applications; This research will provide the scientific foundation 
for decisions on military advertising budgets by the Secretary of Defense. 

Funding; Assistant Secretary of Defense (Manpower) 



FAMILY STRESS AND COPING DURING ROUTINE DEPLOYMENT 
NR 475-031 

Mental Research Institute/Coyne and Nelson 



Scientific Objectives: To understand stress and coping processes in families 
affected by the routine deployment of patrol aviation squadrons in terms of 
the psychological, social, and somatic health of members. To provide the 
foundation for an empirically based, multivariate model of family adaptation 
to deployment and its relationship to service personnel's performance and 
retention. 



AB2££ach: This longitudinal study involves repeated questionnaire assessments 
of 120 families across each of the three phases of deployment. Extensive use 
is made of data concerning individual and family functioning. These include 
measures of stress, coping, psychological resources, informal social support 
and utilization of formal services, and adaptational outcomes, which are 
related to families' attitudes toward the military and to the 
servicemember's evaluations and fitness reports and recommendations for 
promotion. Additionally, these quantitative data are being supplemented with 
qualitative data derived from focus groups conducted with service members and 
tneir spouses. 
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Progress ; All instruments have been developed, pilot tested, and refined, 
and are now in final form. Data collection is continuing, data are beinc 
entered as receive', and codebooks have been developed. Preliminary analyses 
with data from the first waves of assessments have begun, utilizing a multiple 
regression approach to predict satisfaction with military life, morale, and 
commitment to reenlistment from stress and coping variables. The focus 
groups are currently being conducted. 

Potential Applications : Successful completion of this research will give the 
Navy an empirical data base for programs designed to improve organizational 
readiness before, during, and after deployment. Predictors of readiness, 
morale, and commitment to reenlistment be identified, along with sources of 
individual and family vulnerability and strengths. 



EFFECTS OF DEPLOYMENT ON PERFORMANCE 
NR 475-032 

Eastern Virginia Medical Authority/Archer 



Scientific Objectives ; To identify psychosocial responses to deployment 
procedures among Navy personnel and their dependents and relate individual 
and familial response patterns to performance efficiency and effectiveness for 
active duty personnel. 

Approach ; A randomly selected group of 3,500 active duty Navy personnel and 
their dependents is being examined during defined periods of deployment and 
non-deployment in a cross-sectional and longitudinal approach utilizing a 
quasi-experimental design. Data are being gathered across diverse areas of 
adjustment including personality functioning, cognitive functioning, social 
support network characteristics, academic and work performance. 

Pr ogress ; Collection has been completed of pilot data on about 400 Navy 
personnel and 200 spouses, with a questionnaire return rate of about 50%. 
These data are being analyzed and a preliml :ary progress report is being 
prepared. 

Potential Applications ; This research will relate empirical measurements of 
the rate of occurrence of problematic response patterns to deployment procedures 
and will develop a checK list of characteristics of those families who are 
most at r:'.sk for psychosocial stress response and maladaptive functioning in 
response to deployment procedures. 



US MARINE CORPS JOB PERFORMANCE 
NR 475-036 
Texas A&M/Fisher 

Scientific Objectives ; Thf* focus in this research is on understanding the 
cognitive processes by which raters receive, store, recall, and integrate 
information about the job performance of others, and about how individuals 
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self-regulate their behavior based on goal-setting and feedback co them 
from the environment on their past performance. Categorization of a person, 
behavior, situation or object is thought to be the outcome of a 
feature-matching process, in which a stimulus is compared with the features of 
a category "prototype". Once categorization has occurred, the information 
about the stimulus is cognitively integrated with the relevant category 
prototype, and in making subseqnent judgements observers are hypothesized to 
rely heavily on the prototype. Based on these theoretical notions, new 
performance evaluation systems will be designed for the Marine Corps along 
with ne\y systems for training supervisors to make better evaluations. 

Approach ; Marine Corps will define the dimensions on which they wish to have 
people assessed, and Pis will apply their theoretical notions to developing 
job-relevant category prototypes. The key is identifying those behaviors 
which can be considered prototypical for a given location on a dimension. 
This is done by first generating a list of relevant behaviors, then 
separately rating the prototypicality of each behavior. Out of initial long 
lists, only those anchoring definitions are retained which are judged to have 
high prototypicality. These consensus prototypes then become the basis for 
training of performance evaluators. 

Progress ; This work was begun late in FY 85, and no Status Reports have yet 
been issued. 

Potential Applications ; Literature on performance appraisal indicates that 
an improved formal feedback system can result in performance gains of 10 to 
30%. The Marine Corps is presently revising its performance appraisal 
system, and this effort is expected to contribute to that revision. 

Funding ; ONR, USMC 



LONG TERM ORGANIZATIONAL EFFECTS IN NAVY UNITS 

NR AyS'-oa? 

Rensis Likert Associates/Bowers 

Scientific Objectives ; This project builds upon earlier work (NR 475-010) 
which suggested that a ship's culture changes slowly and has a long-term 
impact on performance (even where there has been complete turnover of 
personnel). The goal of this project is to examine these "constancy" effects 
in conjunction with the effects of "personnel velocity" (rate at which new 
recruits move through the system) . The work will augment an already 
substantial data set and will test a proposed explanation for this long-term 
impact. 

Approach ; The method of study involves five steps; (1) develop and validate 
objective indicators of personnel velocity; (2) develop and validate 
objective indicators of individual vulnerability to Project Upgrade actions 
and of psychological and physical escape from a given job situation; (3) 
collect and classify information on discrete events involving each ship 
studied; (4) test the main effects and moderating effects implicit in the 
constancy-velocity model; and (5) calculate the impact of these effects on 
Upgrade discharges and other performance measures. 

86 
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Progress : Work on this project began late In FY-85 and no Statics Reports have 
yet been received. 



Potential Applications ; The earlier findings cited suggest that evaluations 
of ship and crew performance may be systematically overlooking a potent 
determinant (long-term culture effects), which should be understood and taken 
into account. There are implications for management practices, for human 
resource accounting, and for career planning and development. 



PREGNANCY ATTRITION AMONG WOMEN MARINES 
NR 475-039 

Iowa State University/Gerrard 

Scientific Objectives ; This study is designed to determine the antecedents of 
both pregnancy and pregnancy attrition among first-term women Marines, and to 
apply these findings toward future policy-making aimed at reducing pregnancy 
attrition in this population. Data obtained from Marines will be put in the 
perspective of similar data collected from large numbers of civilian women in 
the same age group. 

Approach ; 800-1000 first-term women Marines will be surveyed during recruit 
training, and then again six, twelve, and eighteen months later, with a 
survey instrument covering sexual and contraceptive behavior as well as 
attitudes and knowledge necessary for effective contraception. Comparison 
groups of college women will also complete the survey at about the same time 
to control for variations in cohorts over time. Multidimensional contingency 
table analysis, regression analysis, and multivariate analysis of variance 
will be used to determine the relative contributions of a number of variables 
to effectiveness of contraception and to pregnancy attrition. 

Progress; This work was begun late in FY-85, and no Status Reports have yet 
been received. 

Potential Applications : The first-term attrition rate among women Marines is 
close to 50%, and more than half of this attrition is due to pregnancy. The 
Marine Corps is actively seeking information to guide efforts to design 
interventions to reduce this rate. 

Funding ; USMC 



ARTIFICIAL INTELLIGENCE RESEARCH IN NAVY PERSONNEL ASSIGNMENT 
NR 475-041 

The Turing Institute/Niblett 

Scientific Objectives ; The aim is to produce an expert system which can 
acquire information by general izr-xt .on from expert-supplied examples, and 
which can provide overall system control functions for a large-scale 
personnel assignment system. 
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Approach; A logic programming framework is being used to design, test, 
and implement structure induction methods for the acquisition of expert, rule- 
based knowledge about personnel assignment procedures. These automated 
induction methods will be used to £ cquire a knowledge-base of decision rules 
and procedures directly from an examination of expert behavior. This 
knowledge-base will form an integral part of an intelligent control system 
for a large-scale personrel assignment/distribution system. 

Zrogress: This work began late in 1985, and no Status Reports have been 
issued as yet. 

Potential Applications; The expert system will interface to ?n existing 
system, based on operations-research methods, which is being implemented and 
tested at the Navy Personnel R&D Center. More generally, the method of 
structured induction can be applied to the problem of automatically acquirine 
an expert knowledge-based in any well-defined domain. 

Funding: Manpower Committee 



WORK FACILITATION 
NR 475-042 

University of Maryland/Schneider 

Scientific Objectives; This project is a transition to a 6.2 effort of work 
that was carried out under a 6.1 contract (NR 170-958). The earlier work was 
concerned with conceptualizing and measuring the ways in which management 
behavior facilitates or inhibits employee performance. The model of work unit 
effectiveness developed will now be applied to a Navy environment (OP-OI) , 
and data collected there will be compared with similar kinds of data from 
civilian organizations for an improved perspective on both. 

^EHoSch: Data collection begins with inter^/iew and observational techniques 
to identify the important categories of managerial activities and 
organizational practices in the organizational unit. From these first 
results survey methods are developed and tested for reliability and 

^""^^ members then respond to the survey, and results are analyzed 
with multivariate techniques to test for relationships hypothesized by the 
model. » ^ «r J 

Progress: This work began late iv FY-85, and no Status Reports have yet Leen 
received. ^ 

Potential Applications; Findings will provide a technological base for a new 
system to provide Navy managers with information and recommendations 
potentially useful for enhancing work unit effectiveness. 



ERIC 



88 

96 



DEVELOPMENT OF COMPUTER-BASED INSTRUCTIONAL SYSTEMS FOR TRAINING ESSENTIAL 
COMPONENTS OF READING 
NR 154-519 

BBK Laboratories/Frederickson 



Scientific Ob.lectives ; The objective is to develop and evaluate instructional 
methods for training critical components of reading skill. Computer-based 
instruction incorporates continuous dynamic monitoring and feedback to develop 
and automate efficient decoding and comprehension procedures. A final 
objective is to extend interactive reading theory to include the high-level 
skills needed for analyzing cohesive features of text. 

Approach ; The instructional approach follows an analysis of reading in terms 
of skill components that are functionally interrelated. Components selected 
for training are those that have a broad impact on reading performance and 
that present particular problems for low-ability adult readers. Training is 
component-specific, including immediate feedback on efficiency and accuracy of 
performance. Where there are hierarchical interactions among components, 
training ot lower-order components precedes training of higher-order 
components, and integration of skills is explicitly built into the later 
learning exercises. 

Progress ; Based on related research on controlled vs. automatic components of 
cognitive skill, computer-based systems have been developed for training 
reading-skill components in low-ability adult readers. These training 
systems incorporate advanced graphics, synthesized speech, and Individualized 
instructional monitoring. Graphic feedback on performance and progress is a 
central design feature. Experimental evaluations of the training systems have 
been used to improve their design, as well as to gain more detailed knowledge 
of the structure and acquisition of reading skills. 

Potential Applications ; The instructional methods developed will be useful in 
remedial training programc, and can also be applied in any area of training 
in which automaticicy of skill components is essential to effective high-level 
performance. 

Funding; NPRDC, Manpower R&D Programs. 



A A * * * * 

THE FOLLOWING WORK UNITS WERE COMPLETED DURING THE PAST YEAR: 



AN AUGMENTED COMPUTERIZED READABILITY EDITING SYSTEM 
NR 667-513 

University of Arizona/Kieras 



Scientific Objective? : The objective was to use existing psycholinguistic 
data to develop principles for the design of an automated text-analysis 
system. The system should be able to critique and suggest revisions for a 
typical range of technical text. 
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Approach ; The research plan combined an extensive analysis of the existing 
literature in psycholinguistics with natural language comprehension methods 
derived from artificial intelligence research. The combination of these two 
approaches yielded a principled design for an automated technical writer's aid 
that incorporates a deeper understanding of comprehension and natural 
language structure than do conventional writers' aids. 

Results & Conclusions ; A large number of psycholinguistics articles were 
summarized and an initial set of comprehensibility rules was devised. An 
experiment was improved by using these rules. A demonstration version of the 
comprehensibiJ ity system was constructed using components from earlier 
comprehension models. The demonstration system was able to make many useful 
comments about maintaining coherence and consistent terminology. 

Potentia l Applications ; Development of an actual usable system to improve the 
content of technical material. 

Reports; 

Kieras, D.E. (1985). The potential for advances computerized aids for 
comprehensible writing of technical documents (TR-85/ONR-17) . University 
of Michigan, Psychology Department. 

Kieras, D.E. (1985). Improving the comprehensibility of a simulated 
technical manual (TR-85/ONR-20) . Univerrsity of Michigan, Psychology 
Department. 

Kieras, D.E., & Dechert, Christiane. (1985). Rules for Comprehensible 
Technical Prose; A survey of the Psycholinguistic Literature , (TR-85/ONR-21) . 
University of Michigan, Psychology Department. 

Kieras, D.E. (1985). An augmented computerized readability editing system: 
Final report (TR-85/0NR). University of Michigan, Psychology Department. 



EMPIRICAL STUDY TO ENHANCE THE REENLISTMENT PROCESS OF CIVILIAN PERSONNEL 
WITH PRIOR MILITARY SERVICE 
NR 475-003 

Pennsylvania State University/Stephenson 

Scientifi c Objectives ; The project is designed to clarify the reenlistment 
process of prior-service personnel. The first objective is to utilize a 
market segmentation model to compare and contrast background factors and the 
behavior of new recruits and re-entrants. The second objective is to model 
determinants of relative wage growth, job performance, and time in and out of 
the military. 

Approach: Several methodological approaches were used, including discriminant 
analysis, multiple regression analysis, survival analysis, and contingency 
table analysis. In general, the researchers seek to understand how economic, 
demographic, and military characteristics contribute to an individual's being 
a re-entrant, leaver, or stayer, and secondly, to understand how selected 
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behaviors compare among groups. 

Results and Conclusions ; This project generated multivariate predictive 
models of subgroup membership. Policy conclusions can be inferred from these 
results. For instance, some analysts have viewed prior-service personnel as a 
solution to manning shortages in critical occupations. Contrary to this view, 
the findings suggest that only a few prior-service individuals are desirable 
recruits based on how long they stay once they reenter and how many leave for 
adverse behavioral reasons. 

Pot ential Applications ; Results from, this study have been briefed 
extensively to appropriate Navy offices, e.g.. Chief of Naval Personnel, and 
have led to a number of reenlistment policy changes, all based on empirical 
findings. 

Report ; 

Stephenson, S. P., Mitchell, M.E., Beik, L. L., Macpherson, D, A., Fitch, S. 
D & Ellison, D. R. (19d5) An empirical study to enhance the reenlis tment process 
of civilian personnel with prior military service (TR-ONR-85-1) . Pennsylvania 
State University, Institute for Policy Research and Evaluation. 



IMPROVING THE EFFECTIVENESS OF NAVAL RECRUITING EXPENDITURES FOR QUALITY 

ENLISTMENTS 

NR A75-00A 

Duke University/Morey 

Sci entific Objectives ; To improve the budget building (POM) process where 
monies for recruiter field level support and Navy specific advertising must 
be defended, given enlistment objectives, number of recruiters to be made 
available, the prospective recruiting environment, and the level of DoD 
Joint Advertising. The capability for performing extensive sensitivity 
analyses on the budget, needed as various of the above considerations 
are varied, was also highly desired. Also of interest were improved 
enlistment goal-setting and recruiter-allocation procedures. 

Approach ; Econometric models which take advantage of natural variations 
in support level per recruiteis, number and location of recn-iters , 
advertising, etc. were used to estimate key elasticities. Phase models 
were first validated on separate years of data to establish their 
reasonableness and credibility. Then the resulting budget generation 
formulae were programmed for an IBM-PC. 

Results & Conclusions ; A fully tested and documented IBM-PC program was 
installed at Naval Recruiting Command Headquarters, complete with user s 
manuals and examples. In exercising the program for recent years, the 
program gave results within a few percent of that actually spent. In 
addition to providing the ideal tradeoff %etween recruiter support and 
advertising (by area by quarter) , the program is capable of handling contract 
goal-setting and geographical recruiter allocation In several different fashions. 
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Finally by incorporating the pay for recruiters it can help determine the 
cost-effective number of recruiters that should be made available. 

Potential Application; The program can quickly perform sensitivity and 
what-if" type analysis. A host of scenarios can be analyzed, e.g., 
pessimistic and optimistic forecasts of the unemployment rate, contract 
objectives, etc. The program can analyze impacts of changes in military pay 
versus civilian pay. 

Report ; 

Morey, R. (1985). Trading off recruiter field level support versus 
advertising; A budge t building model incorporating Optimized recruiter 
allocatio n and contract goalinp (Technic- i Report ONr-389-2) , Duke 
University, Fuqua School of Business, April 1985. 



Archival Publication: 



Morey, R. (1985). Managing the Armed Services Delayed Entry Pools to 
Improve Productivity in Recruiting, Interfaces , L5 (Sep. /Oct.). 



TRANSITION SOCIALIZATION EFFECTIVENESS IN ORGANIZATIONS 
NR 475 -019 

Texas A&M University/Fisher 



Scientific Objectives; To increase our understanding of the socialization 
and adjustment processes that occur during job transfer. 

Approach: Approximately 150 NCOs participated in the primary research 
project. All were interviewed prior to moving, and were surveyed three 
months after arriving at their new location. Most previous transfer research 
has been cross-sectional and retrospective in nature, whereas the present 
study is longitudinal and predictive. Dependent variables to be predicted 
were post-transfer performance, adjustment, and attitudes. Independent 
variables include transfer history (number and frequency of moves), family 
size and age variables, cost of the move, subjective difference between old 
and new locations ("contrast"), realism of pre-move expectations 
( surprise"), and job/career variables. 

Results a nd Conclusion ; Post-move adjustment indices were found to be 
predicted by pre-move attitudes toward the move, adjustment and satisfaction 
with previous moves, number of dependents, unmet expectations, perceived job 
advancement, and amount of information about the new location received prior 
to moving. Time to "get up to speed" in a new job and both performance and 
role ambiquity three months after moving were strongly predicted by the 
similarity between the old and new jobs. 

Potential Applications; The research suggests that transfers will be more 
successful if transferees perceive the move as a job or career advancement, 
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develop positive expectations before the move, and are supplied with information 
about the new job and location prior to moving. When individuals are 
transferred to dissimilar jobs, they should remain in the assignment several 
months longer than usual in order to maintain the same ratio of learning time 
to productive time as is realized when transferring between similar jobs. 



Reoorts : 



Shaw. J. B., Fisher, C. D., and Woodman, R. W.(1985). Predictors 

nf transf er adjustment; A longitudinal study (TR-ONR-7). Texas A&M 

University, Department of Management, January 1985. 

Jones, G. R. and Fisher, C. D. (1985). The relationship between 
performance and satisfact ion- A utility analysis (TR-ONR-8). Texas A&M 
University, Department of Management, March 1985. 

Shaw, J.B., Fisher. CD., and Woodmar R.W. (1985). Making a transfer; An 
analysis of qualitative data relevant to tranfer attit udes and adiustment 
(TR-ONR-9). Texas A&M University, Department of Management ^ June 1985. 

Fisher, CD., Shaw, J.B., and Woodman, R.W. (1985). Final report on transition 
socialization effectiveness; Year three (TR-ONR-10) . Texas A&M University, 
Department of Management. 
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PERSONNEL AND TRAINING RESEARCH PROGRAMS 



(CLUSTERS J-M) 



en^hJ^h!"""''? ^''^ Training Research Programs support research which will 
enable the naval services to more efficiently acquire, train, and use 

^ZllT ' °" "^^"^"^ are'focu^ed on ^e'ods or 

assessing people's potential arid competence and for conducting rapid, 

tiir t ^ Two considerations guide our effort. First, it appears 

Irti.^^^ best opportunities for increasing the efficiency of assessment and 
iImI^A^ J" l"" individualizing (usually using computers) each process to each 
i^M^^ . 'fr'' increasingly technological character of many 

tlltnlt^^ t """i"^' ""^^"^ emphasis. These considerations have led us to 

I^JZ^m'"" °n objective, quantifiable theories of cognition and 
NatuSlJi°''"'T'r performance of the types required on Navy jobs, 
needed to' "''""^ concerned with the methodologies 

Tolnitill '"m^ . . °f assessing individual!' 

trfJn ;r I and competencies, and improving those competencies through 

cluster!: theories and methodologies falls into four main 

, ^' Theory-based Personnel Assessment. Research in theory-based personnel 
assessment should lead to more efficient and valid psychological assessments 
fuLtiSL ' ^"'^ selection, classification .'advancement . and tr'n^ng 

of .!]i.l!t■^T^':°""^''°"'!'"^^^^^^'^""• Action . Exploration 

aLj 1 '"'"T'' parameters of information processing is 

N^r^ jobs? understanding of aptitudes and other abilities important in 

theor;..^?nn^r^^°"^^ ^"'^ Terhnmof.y. This work on instructional 

theories looks to eventual application in generative, knowledge-based. 

deieTo^l^/^'r"" J"'''^''^ cognitive skills, and to improved methods for 
developing and evaluating training materials. 

Cognitive Processing. This last cluster is aimed at models of the 
cognitive processes and structures underlying skilled performance in real- 
world tasks with complex information-processing demands. 
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J. THEORY-BASED PERSONNEL ASSESSMENT 



The measurement of individual differences is an indispensable part of 
psychological research, both theoretical and applied. It is the main source 
of data in studies of the evolution of both cognitive and non-cognitiva 
competence and performance and in other areas of psychology which seek to 
explain individual behavior. It is the basis for the Navy s personnel 
selection and classification technology, for career counseling, for the 
design of instructional interventions, and for performance evaluation. Its 
role is central whenever a case-by-case approach is taken to personnel 
decisions and to behavioral, instructional, and performance-aiding 
interventions of all kinds. The primary impact of the work described in this 
cluster it to enhance the accuracy, efficiency, validity, and scope of the 
measurement of individual differences. 

Work in this cluster is attacking fundamental measurement issues form a 
variety of perspectives, including: (a) theoretical development of the 
measurement foundations, (b) mathematical and numerical development of 
modelling techniques, and (c) experimental and simulation studies of assess 
model adequacy. Work is focused on developing and extending item-response 
theory, a powerful unified framework for relating behavior to unobservable 
traits Within this framework rigorous mathematical formulations are being 
developed for assessing whether two measurement paradigms are measuring the 
same psychological trait(s), for discovering the shapes of "^"'-"^P""^^ . ^ 
functions without strong a priori shape constraints, for modelling behavior 
on groups of tasks which falTT^be conditionally independent, for calibrating 
item-response functions from the sparse data available during operational 
test, and for identifying examinees for whom the models fail to fit. 



Collaboration among investigators in this cluster, and coordination with 
other agencies, is extensive. Four work units involve collaboration among 
researchers at multiple institutions. ONR holds is a member of the 
Computerized Adaptive Testing/Armed Services Vocational Aptitude Battery 
(SI?-ASVAB) WorkLg Group. In direct support of f^-ASVAB Working Group 
mission, one work unit (NR 4421-520), jointly funded ''Ith AFHRL, NPRDC, 
MEPCOM, and OASD(M&RA) is exploring techniques for the calibration of 
item-response functions on-line during computerized ^^^f ^^^^ ^"^^"f ' J" 
support of Navy and Marine Corps training, two work units (f j^^jl-''^^ & 
4422-531) are estfiblishing research laboratories in Navy and Marine Corps 
pchools as test beds for model-based measurement concepts. We also hold 
annual contractors' meetings focusing on model-based measurement issues. 
These meetings typically attract all of the leading contributors to model- 
Jased measurLenfand include representatives of other Navy, DoD, and civi lan 
federal agencies. These meetings serve to foster mutual 3xchange of recent 
insights and research results. 
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LATENT-TRAIT THEORY FOR DISCRETE-RESPONSE MODELS 
NR 150-467 

University of Tennessee/Samej ima 



Scientific Objectives; This work builds upon Samejima's earlier research on 
function estimation for nominal, graded, and continuous responses! and L 
mult -dlnenslonal latent spaces. The current work has three majo^ components- 
va) further refinement and evaluation of methods for estimating °"»P°"e"ts. 
response-category operating characteristics for multiple-choice items; (b) 
theoretical, statistical, and numerical development of a class of models to 
replace existing three-parameter models for multiple-choice tests- and fc) 
structuring of the field and the preparation of integrated w^itt^; Ztelllu 
and computer programs which make recent develoments useable for applications. 

to£5£h: Refinements of two estimation procedures are being explored. 

! ? fuf".^^^''-^ "^^"^ maintain the richness of the class of 

admissible functions and minimize estimation instability. Further theoretical 
development of Samejima's model for multiple-choice test items is focusing 
?Sr combi;in^"% dlstractor quality and on techniques 

^rLSf m\ ? ^J^" ^^«ily useable computer 

programs which implement theoretical results are being developed. 

St?7^ ^r^^i^rf ^"'^ approaches that have been 
developed and modified m this research for estimating the operating 
characteristics of discrete responses work well. The operating characteristics 
of the wrong alternative answers of multiple-choice tesE items! whichire 
named plausibility functions. Many of them turned out to be quite informative 
In other work, the effect of assuming the three-parameter logistic mode^or 
simulated data which actually follow the (two-parameter normll) given mod Swas 
investigated. Results clarified its extent of danger and tolerance 

lllTn^^^ Applications ; This work is providing theoretical underpinnings for 
funct?iir r^''' "f;. investigating the shapes of item-response 

functions, (b on-line calibration of new test items from the data collected 
during operational tests, (c) vertical equating of tests intended for 
broad-range tests, (d systematic analysis of the quality of dlstractors on 
multiple-choice test items, and (e) modelling response data which Includes 
both continuous and discrete aspects. 
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Reports ; 

Samejima, F. (1935) P lausibility functions of Iowa vo cabulary test Items 
estimat ed bv the slmplp sum procedure of th e conditional pdf approach 
(Technical Report ONR-84-1). Knoxvlile, TN: University of Tennessee, 
Department of Psychology. December, 1984. 

Sameilma, F. (1985) Comparison of estimated Item parameters of Shlba's 
word/phrase c omprehension tests obtained by LOGIS T 5 and by the tecrachorlc 
method (Technical Report ONR-84-2). Knoxvllle, TN; University of Tennessee, 
Department of Psychology. December, 1984. 

Sameilma, F. (1985) Results of item-parameter estim ation using LOGIST 5 on 
simulation data (Technical Report OKR-84-3) . Knoxville, TN: University of 
Tennessee, Department of Psychology. December, 1984. 



FORMULA-SCORE THEORY 
NR 150-518 

University of Illinols/Levine & Linn 

Sc ientific Oblectives ; This work is more fully developing the Formula-score 
Theory (FST) approach to modelling performance on psychological tests. Within 
that formula-score framework it is developing techniques for gaugirg whether 
two psychological tests measure the same capabilities, for analyzing the 
relative contributions of well-defined capabilities to test performance, for 
describing the shapes of item-response functions, and for detecting changes in 
those item- response functions. In addition this work is exploring fundamental 
issues on equating multidimensional adaptive and conventional tests, on 
maximal curves for gauging the limits of appropriateness measurement 
procedures, and on the measurement of item bias in subpopulatlons. 

Appro ach; Through a combination of theoretical and empirical studies the 
investigators are (a) refining procedures for obtaining and evaluating the 
quality of a canonical representation of ability distributions, (b) 
formulating procedures for measuring the equivalence of two capability 
distributions or two item-response functions, and (c) developing techniques 
for discovering the shapes of multidimensional item-response surfaces. In 
addition empirical studies will gauge the seriousness of drift in an equating 
between a multidimensional conventional and adaptive test as a result of 
successive replacement of items in the adaptive pool. 

Pr ogress ; Major completed applications of FST Include: (a) determining the 
shapes of item-response functions for a conventional unldlmenslonal test, (b) 
providing a polychotomous model and accurately estimated response functions 
for the wrong responses and for omissions on a unldlmenslonal test, (c) 
efficiently estimating unldlmenslonal ability densities from small samples 
without assuming a parametric form, (d) Increasing the reliability of scores 
on a conventional test by providing an ability estimator with the same 
conditional expected value and smaller conditional variance than number-rignt 
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for examinees at every ability level. In addition progress has been nade in 
developing a theoretical framework for (a) gauging the extent to which uwo 

discovering how two abilities interact to 

illlill Hil'^H JT""" r "'^ """^ estimating a two-dimensional 

ability distribution without the usual strong shape constraints. 

Potential Applications; The availability of Formula-score Theory (FST) 
promises to provide the theoretical foundations for improvements in Navy and 
Marine Corps measurement activities of all types. Formula-score theory 
represents the functions needed to analyze a test as a linear combination of a 
small number of functions intrinsic to the test. The representation's 
simplicity encourages novel solutions to standard problems. For example, 
instead of transforming conventional scores on a new test to make then 
comparable to scores on an old test, FST shows how to compute a new score 
airectly from the new test answer patterns. FST's linearity expedites 
reformulation of hard measurement problems (e.g., ability distribution 

r "'^"f^^J mathematical problems (maximize a convex function), 
and facilitates analysis of unidlmensional tests and complex test batteries. 

Reports ; 



Drasgow, F. 

scores 



'(rl'J l^^^^'lk "^'^ "'i^^^^^- O ptimal detection of inappropriate r.. r 
Measure^e:t^'Lab:?;tory":^- IL: University of IlliJSis^Model-baled 

Levine, M. V. & Drasgow. F. (1984). Performance envelopes and optima al 

appropriateness measurement (Technical Report 84-5) . Champaign IL- 

University of Illinois, Model-based Measurement Laboratory. ' 

Archival Publications ; 

Drasgow, F Levine M. V.. & Williams. E. A. (1985). Appropriateness 

BritJsTT^LL "sponse models and standardized indices. 

British Journal of Mathe matical and Statistical Psvcholof>v . 38, 67-86. 

DIMENSIONAL MISSPECIFICATION AND APPLICATIONS OF ITEM-RESPONSE THEORY 
University of Iowa/Wang 

^-uTfT^rS^^f^' °^ ^""""^ theoretical foundations for 

multidimensional item-response theory, ther3 has been considerable confusion 

rnn^p% ^'i? researchers have equated psychometric dimensions with 

Z J^V?. '^}'"^"^^°"«' that a test problem which requires more than 

one ability (e.g., reading and arithmetic ability) necessarily requires a 
multidimensional model. Others have called upon the notion of a "complete 
]rlZ to argue that a test model is inappropriately dimensioned if 

items appear to be conditionally dependent. Still others have called on the 
for V f inappropriate, factor analytic conceptualization 

bv orJ!-Jj/ .\ '^^^ ""''^ ^^^^ contribute to improving that situation 

by providing a theoretical framework for addressing dimensionality issues 
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within a broad class of models. 



Approach ; A mathematical framework for representing the multidimensional 
situation for a broad class of item-response-theory models is being developed 
and v'ill be used to anticipate the implications of multidimensionality for 
model calibration, for adaptive testing, for test bias, and for test equating. 
In addition, improvements to multidimensional modeling and adaptive technqiues 
are being explored. 

Progress : An analytical framework based on the linearity of the item logic 
scores for the usual compensatory logistic item response model was defined. 
Within that framework algebraic relationships between unidimensional estimates 
and true multidimensional parameters were derived. Analytical results were 
demonstrated with simulated data. Finally, an index of the dimensionality of 
the latent space was suggested. 

Potential Applications ; Unidimensional models are currently the only 
practical vehicles for item-response-theory-based test development, 
administration, and scoring applications. Yet most useful item pools are 
multidimensional. Among other things this work will provide the theoretical 
wherewithal for addressing issues such as robustness ard bias in specific 
applications. 



PSYCHOLOGICAL TEST DIMENSIONALITY AND BIAS 
NR 150-533 

University of Illinois/Humphreys, Stout & Tucker 

Scientific Objectives ; The standard method of testing the dimensionality of 
binary items sets is to compute tetrachoric correlations, obtain the 
principal components, and Inspect the latent roots. Unfortunately, 
tetrachorics have undesirable properties which limit their usefulness in 
practice; the matrix of tetrachorics is often non-Gramian and the sampling 
errors of the tetrachorics vary widely as a function of the difficulty levels 
of the items correlated. The objective of this work is to develop and compare 
Improved methods for assessing dimensionality of binary tests. 

Approach : The basic approach involves theoretical development of the various 
methods, analytic and Monte Carlo techniques for examining their statistical 
and numerical properties, and empirical studies to check results. 

Progress : (1) A high-fidelity simulation rtodel has been constructed which 
replicates the essential characteristics of typical cognitive data sets; (2) 
Empirical studies compared several indices based upon eigenvalue differences, 
patterns of factor loadings, and measures of conditional independence. While 
none of the indices was wholly satisfactory, the index based upon conditional 
independence faired better especially with ]arge sample sizes, large numbers 
of items, narrow ranges of item difficulty, and when distinguishing between 1- 
ar^ 2-dimensional data; (3) A Monte Carlo study of Stout's test of 
unidimensionality has been completed. Results indicate that when combined 
with Stout's bias correction procedure, the test performs well for the test 
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lengths and population sizes typically encountered in practice; (4) A 
mathematically rigorous notion of the "essential" dimensionality of a test has 
bien developed, which captures the reality that performance on test items is 
multiply determined. A theoretical analysis indicates that Stout' s test has 
good power against "non-essentially" unidimensional alternatives; and (5) A 
rigorous notion of test bias has been framed in terms of the essential 
dimensionality of the test in certain subpopulations. Based upon this notion 
a statistical test of test bias has been formulated and is currently beine 
evaluated. ' * 

Potential Applications; In adaptive testing different examinees answer 
different sets of test items. In such circumstances, arbitrarily selected 
sets of items must measure the same essential dimension. Methods under 
development in this project will heJp test constructors insure that this is 
the case. 



FULL-INFORMATION FACTOR ANALYSIS OF THE CAT-ASVAB ITEM POOL 
NR 150-541 

National Opinion Research Center/Bock 



Scientific Objectives; This work involves (a) improvements to the Bock-Aitkin 
algorithm for full-information factor analysis, (b) application of the 
procedures to assess the dimensional structure of the item pools developed for 
operational use of the computerized adaptive version of the Armed Services Vocational 
Aptxtude Battery fCAT-ASVAB), and (c) the development of practical strategies 
for dealing with multidimensionality in CAT-ASVAB subtests. 

Approach; Empirical studies of the dimensional structure of the CAT-ASVAB 
item pools are being conducted for explanations of obtained multidimensionality. 
Lmpirical studies are examining the performance of several techniques for 
the modeling of guessing on psychological test items. Techniques are also 
being explored for adaptively administering multidimensional subtests. 

Progress; The first phase of this effort focused on extending the 
full-information factor analysis procedures to include estimation of lower 
asymptote parameters and new and faster numerical methods for evaluating 
marginal likelihoods. To avoid the problem of fitting a "guessing factor" 
investigators tested a one-dimensional model with a non-zero lower 
asymptote parameter before increasing model dimensionality. After the number 
ot dimensions was determined, improvements to the estimate of the 
lower-asymptote parameter were sought. To speed up calculations, investigators 
explored the use of "adaptive" quadrature; the likelihoods were evaluated 
?f "^^''^ "^^^^ appreciable posterior probability. These points were 

identified from each examinee' s estimated factor scores and their standard 
error of measurement. 

Potential Applications; Several factors underscore the importance of this 
work to the Services; First, adaptive testing Is not appropriate with 
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seriously multidimensional item pools. Second, the Bock-Altkin algorithm is 
the only well developed technique for assessing multidimensionality in an 
itera-response-thcory sense which assigns items to dimensions. Third, 
applications of the algorithm to the Profile of American Youth y^SVAB data set 
suggested serious multidimensionality in certain subtests. Fourth, there is 
a possibility that some of these additional dimensions may not show up in the 
somewhat narrower nilitary applicant population. Fifth, there iq a possibility that 
some of the additional dimensions are artifact ; of the inadequacies of the BocV- 
Aitkin algorithm for the modelling of guessing. Finally, inasmuch as 
substantial further subdivision of ASVAB into more nomogeneous classes is 
impractical, alternate adaptive strategies are required. 

Funding : Air Force 



MODELLING CONDITIONAL DEPENDENCIES AMONG PSYCHOLOGICAL TLST ITEMS 
NR 150-538 

University of Illinois-Chicago/Gibbons 

Scientific Obiectives : An assumption of conditional independence has been 
commonplace in theoretical and practical work in item-response theory. The 
assumption states that item scores are related to each other only through 
their relationship with examinee ability. This means that item scores are 
statistically independent for examinees with the same ability. This work Is 
developing techniques (a) for gauging the extent of conditional dependencies 
among test items, (b) for obtaining efficient ability estimates In spite of 
minor dependencies, and (c) for explicitly modelling dependencies in response 
data. 

Approach : Initial work is focused on theoretical and numerical development o: 
the techniques. Subseque:-*:ly , techniques will be validated with Monte Carlo 
simulation and with actual data sets. 

Progress : This work began late in 1985, and no Status Reports have yet 
been issued. 

P otential Applications : (a) Substantial evidence suggests that conditional 
dependencies exist in many Navy and Marine Corps testing applications. These 
include, for example, multiple questions about a single paragraph in 
paragraph-comprehension test Item, about a single schematic In items 
on a electronics test, or about a single problem situation In a 
problem-solving task; (b) Extant technology either for identifying or for 
dealing with conditional dependencies when they exist is wholly Inadequate,; 
(c) Ignoring conditional dependencies when they exist at b.:;st decreases the 
efficiency of ability estimation and at worst leads to the wrong conclusions. 
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ON-LTNE CALIBRATION FOR THE ADAPTIVE ASVAB 
NR 150-520 

Educational Testing Service/Lord, Stocking, & Holland 
Univerfilty of Chicago/Bock 
University of lUinois/Levine 
University of Tennessee/Saniejima 

Scientific Objectives : The calibration of replacement test items for the 
adaptive Armed Services Vocational Aptitude Battery (ASVAB) requires 
techniques for accurately estimating item-response functions from the sparse 
dpta sets available dur^ng administration. It is the objective of this work 
unit to develop and evaluate techniques for that purpose. 

Approach ; Four methods of on-line item calibration are being formulated and 
compared. These methods are extensions of the principal off-line calibration 
methods and Include techniques based upon marginal maximum-likelihood (see NR 
150-5A1 and A75-018), joint maximum-likelihood, formula-score theory (see NR 
150-518), and Samejima' s non-parametric approach (see NR 150-A67)/ The 
results of the application of each technique will be fitted three-parameter 
logistic item-response functions. Performance of the methods will be compared 
using common criteria and simulated ASVAB data sets. 

Progress; Substantial progress has been made in extending and improving the 
four calibration techniques. Further improvements and testing with Monte 
Carlo studies are in progress. In addition, rudimentary techniques for 
modelling trends in item parameters have been developed and tried 
out on real data sets. Analyses suggest that changes in item difficulty do 
occur, but are slow and regular. Improvements to the procedure and techniques 
for appropriate compensatory adjustments are under study. 

Potential Applications : On-line calibration techniques are essential for the 
anticipated adaptive ASVAB, Initially, these techniques will be needed to 
check calibrations obtained from studies in which items were administered 
under experimental instructions and in paper-and-pencil format, Subsequently, 
these techniques will be used to calibrate replacement items and to maintain 
scale integrity. 

Funding ; ONR, AFHRL, MEPCOM, NPEDC, OASD (M6RA) 
Archival Publications; 

Lord, F, M, (198A), Standard error of measurement at different ability 
levels. Journal of Educational Measurement , 21 , 239-243, 

Lord, F, M, (1985), Estimating the imputed social cost of errors of 
measurement, Psychometr ika , 50, 57-68 

Lord, F, M,, b Wingersky, M, S, (198A), Comparison of IRT-true-score and 
equipercentile-observed-score "equatings", \pplied Psychological Measurement, 
8, 453-461, ^ ^ 
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Samejima, F. (1984). Review of Doignon, J. P. and Ducamp, A., On the 
realizable biorders and the biorder dimension of a relation. Jo urnal of 
Mathematical Psychology , 28, 73-109. 

Wingersky, M. S., & Lord, F. M. (1984). An investigation of methods for 
reducing sampling error in certain IRT procedures. Applied Psycholoical 
Measurement , 8, 347-364. 



THE INTERPRETATION AND APPLICATION OF MULTIDIMENSIONAL ITEM-RESPONSE THEORY 

MODELS 

NR 150-531 

American College Testing Program/Reckase & Carlson 

Scientific Objectives ; This work is developing useable multidimensional 
item-response theory (MIRT) for training applications. Various lines of 
work are exploring (1) the adequacy of compensatory models for characterizing 
typical training performance; (2) techniques for increasing the accuracy and 
efficiency of item-response-function-estimation algorithms (especially in small 
samples); (3) extensions of the unidimensional notions of item difficulty, 
discrimination, and information to a general class of MIRT models; (4) the 
implications of adaptive item selection for the complexity of the content 
dimensions being measured; (5) the effectiveness of appropriateness indices 
for detecting items whose characteristics have changed since the response 
models were originally calibrated; and (6) the nature of and effective methods 
for dealing with the trade-off between test efficiency and test security in 
applications. 

Approt ch ; The shapes of theoretically and empirically determined 
item-response functions are compared tc investigate the adequacy of the 
more tractable compensatory models. Bayesian procedures are explored to 
increase the efficiency of multidimensional-item-response-function e timatlon 
in small samples. The theoretical relationship between multidimensional 
difficulty and a local notion of item information is studied. Reckase' s 
previous work on the relationship between target and obtained content of tests 
which have been optimized via item-response theory is extended (see NR 
150-474). Finally, results are validated in the Marine Corps Communications 
Electronics School at Twenty-nine Palms, CA. 

Pr ogress ; A definition of the difficulty of an item which measures more than 
one dimension has been formulated. This definition represents an item's 
difficulty as an n-dimensional vector, where n is the number of skills being 
tapped. The unidimensional notion of an item information function was 
extended to the multidimensional case. This extension indexes the extent to 
which performance cn the item differentiates among examinees with particular 
combinations of skills. A detailed study of how multidimensional difficulty 
and discrimination can be used to form item sets which operate as if they were 
unidimensional has been conducted. The hypothesis was that a set of items 
which have their highest discrimination in roughly the same direction will 
operate as if it were unidimensional. Analyses of data from a mathematics- 
usage test generally supported this hypothesis, although some small item 
sets which were fairly difficult were problematic. 
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Potential Applications ; This work addresses several Important issues in 
multidimensional Item-response theory: model specification, model estimation, 
and test construction. Progress on each of ttiese issues 1.- Important if 
multidimensional item-response theory is to be made practical, for example, 
with large item pools or with small samples. 

A rchival Publications ; 

Reckase, M. D. (1984). Review of "Applications of Item-response Theory" by R. 
K. Hambleton (Ed.). Educational Measurement ; Issues and Prac tice. 3, 26-27. 

Reckase, M. D. (1984). Scaling techniques. In G. Goldstein and M. Herson 
(Eds.), Handbook of psychological assessment . New York: Pergamon Press. 

Reckase, M. D., (1985) The difficulty of test items that measure more than one 
ability. Applied Psychological Measuren-^nt , 9_, 401-412. 

Reckase, M. D. & McKlnley, R. L. (1985). Some latent-trait theory in a 
multidimensional latent space. In D. J. Weiss (Ed.), Proceedings of the 1982 
item-response theory and computerized adaptive testing conference . 
Minneapolis, MN: University of Minnesota, Department of Psychology. 



A DISCRETE LATENT STATE APPROACH TO DIAGNOSTIC TESTING 
NR 150-466 

Portland State University/Paulson 




Scientific Oblectives; The purpose is to Improve the measurement foundations 
of tests for mastery of specific concepts. The general principle guiding the 
effort is that the characterization of knowledge states should be derived from 
psychological theory and that the measurement model should relate performance 
to those knowledge states. 

Approach; Latent class models are explored which represent examinees as 
being in one of a small number of states. Estimation procedures are 
developed to estimate model parameters. Goodness-of-f it statistics are 
developed to assess the likelihood of the model given the data. The 
performance of the procedures are examined using data on slgned-number 
arithmetic . 

Progress; (1) Estimation procedures for model parameters based on the EM 
algorithm were developed. These procedures substantially Increase the 
number of items which can be handled simultaneously by the latent-class 
models. (2) Indices of model fit based on a likelihood ratio statistic have 
beon developed. (3) Empirical studies aimed at modelling slgned-number- 
arlthmetlc performance were completed. Results indicated that while 
complex models are needed to diagnose many of the student misconceptions, 
simpler models do surprisingly well. (4) A test for dimensionality based 
upon so-called "monotonic homogeneity" considerations was suggested. 

Potential Applications: In many Navy and Marine Corps training contexts, 
instructors have explicit notions of the typical misconceptions which occur at 
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various points in the instructional sequence. In those circumstances these 
models provide the theoretical wherewithal to relate performance data to 
specific misconceptions. 

Report ; 

Paulson, J. (i985). Latent class representations of systematic patterns in 
test responses (Technical Repurt 85-1). Portland, OR: Portland State 
University, Department of Psychology. 



TECHNIQUES FOR DLAGNOSING ERRORS ON PROBLEM SOLVING TASKS 
NR 150-495 

University of Illinois/Tatsuoka, Tatsuoka, & Eddins 

Scientific Obiectives ; To develop a stochastic model that is capable of 
diagnosing and classifying cognitive errors in well-defined procedural domains 
to apply the model to detect weaknesses in instructional materials, to 
optimize remedial procedures, and to evaluate psychological models. 

Approach ; A theoretical framework for diagnosing errors in procedural skills 
is refined by improving the parameter estimation and pattern classification 
technologies and by incorporating measures of error consistency and response 
time. Work aimed at modelling transitions of error types as a trainee 
progresses through a course of instruction explores the use of Markov 
chains. A general method of item construction for tests of procedural 
knowledge is sought. Finally, the generalizability of results is 
checked in a Navy Basic Electricity and Electronics school. 

Progress : Accomplishments include: (1) "Rule Space", which is the Cartesian 
product of an examinee's trait standing and a measure of the consistency of 
h, is/her response pattern with the model, was refined and tried out in a 
variety of domains. Results confirmed its ability to diagnose well-defined 
procedural errors in signed-number arithmetic, arithmetic with fractions, 
English srammar, and algebra. (2) Previous studies indicated a strong 
relationship between the consistency with which trainees apply their rules and 
their level of expertise in a procedural domain. To gauge consistency, 
properties of the determinant of the covariance matrix over parallel subtests 
were examined. Experimental studies comparing the consistency of groups 
taught by different methods suggested that the method with mote consistent 
rule application had a higher retention rate one year later. (3) Techniques 
for gauging the seriousness of incorrect rules using the norm-conformity index 
have been formulated. (4) A new adaptive procedure has been designed. 
Essentially it is a multivariate, min5mum-variance optimization procedure and 
yields substantial savings in the number of items needed for a diagnosis. 

Potendal Applications : This project has four broad areas of application. 
First, it is exploring quantitative methods for assisting the instructional 
designer in studying the structure of a domain to be learned, in identifying 
weaknesses in existing instructional materials, and in selecting test 
questions to make efficient use of testing time. Second, it is exploring a 
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variety of techniques for specifying, estimating, and evaluating performance 
models which will enable more specific diagnoses and which will serve ao the 
basis for the prescription of remedial materials. Third, it is exploring 
quantitative techniques for studying the seriousness of particular 
misconceptions as a basis for deciding whether to remediate them before moving 
on to new material. Finally, it is exploring rhe problem of communicating 
often abstract diagnoses to instructors and to trainees in ways that will be 
useful to vi'icm. 

Report^: ; 

Tatsuoka, K. K. (1985). Diagnosing cognitive errors; Statistical pattern 
classification and recognition approach (Research Report 85-1-ONR) . Urbana, 
XL: University of Illinois, CERL. 

Tatsuoka, K. K., & Tatsuoka, M. M. (1985). Bug distribution and pattern 
classfication (Research Report 85-3-ONR) . Urbana, XL: University of 
Illinois, CERL. 

Tatsuoka, K. K., & Yamamoto, K. (1985). Application of component scoring 
to a complicated cognitive domain (Research Report 85-2-ONR) . Urbana, XL: 
University of Xllinois, CERL. 

Archival Publications ; 

Birenbaum, M., & Shaw, D, (1985). Task specification chart: A key to a better 
understanding of test results. Journal of Educational Measurement , 22, 

Morimoto, Y., & Tatsuoka, K. K. (1984). Analysis of misconceptions in fraction 
problems: Interactive diagnostic system on the PLATO system. Xn The 
Proceedings of ICMX-JSME Regional Conference on Mathematics Education, Tokyo. 

Tatsuoka, K. K. , (1985). A probabilistic model for diagnosing misconceptions 
in the pattern classification approach. Jour nal of Educational Statistics, 10, 
55-73. 

Tatsuoka, K. K. (1985). A latent trait model for interpreting misconceptions in 
procedural domains. Xn D. J. Weiss (Ed.), Proceedings of the 1982 Item 
Response Theory and Computerized Adaptive Testing Conference Minneapolis, 
MN: University of Minnesota. 

Tatsuoka, K. K. (1985). Review of Test of English as a Foreign Lanaguage . 
Lincoln, NE: The University of Nebraska, The Buros Institute of Mental 
Measurements . 

Tatsuoka, K. K. (1985). Review of thn Test of Spoken English . Lincoln, NE: 
The University of Nebraska, The Buros Institute of Mental Measurements. 

Tatsuoka, K. K., & Eddins, J. M. (1985). Rule space. In S. Kotz & N. L. 
Johnson (Eds.), Encyclopedia of Statistical Sciences , Vol 2 . New York: Wiley. 
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WORKSHOP AND CONFERENCE ON DIAGNOSTIC MONITORING OF KNOWLEDGi: AND SKILL 

ACQUISITION 

NR 667-553 

Educational Testing Service/Frederiksen 



Scientific Objectives : To foster new thinking end ultimately new approaches 
for representing and diagnosing trainee knowledge and skill as it evolves in 
the course of instruction. 

Approach : A group of 18 innovative researchers from the fields of artificial 
intelligence, cognitive psychology, and psy^hometrics will neet to discuss the 
difficult theoretical and practical problems of automated cognitive diagnosis. 
An initial meeting was held to define the focus of the participants' efforts, 
to define areas of common interest, and to establish collaborative 
relationships among some of the researchers. At a second formal conference, 
participants will present the results of their efforts. A proceedings will be 
published in book form. 

Progress : This work began late in FY 85, and no Status Reports have yet been 
issued. 

Potential Applications : Experimental, mathematical, and computational 
investigations of complex knowledge structures and their relation to behavior 
are central to the Navy and Marine Corps personnel and training research 
interests. This effort will result in improvements in intelligent training 
systems and Al-based classroom aids. 



BAYESIAN TECHNIQUES FOR ITEM-RESPONSE THEORY 
NR 150-535 

University of Missouri-Columbia/Tsutakawa 

Scientific Objective s; In item-response theory (IRT) an examinee' s item 
responses are linked to an ability scale through item-response functions 
(IRF). In the typical IRT application, these IRFs are partially known 
through a calibration based on examinees who have previously taken the 
same test. The typical application proceeds as if these IRFs were 
well-estimated; in effect this assumed that the calibrations are exact. In 
previous work Tsutakawa developed techniques for representing uncertainty 
about the calibrations of these IRFs. In this project that representation is^ 
checked and used to more accurately represent what is known about an examinee's 
ability. Subsequently, this improved representation of an examinee' s ability 
will provide better estimates of IRFs for new, uncalibrated items. 

Approach; This work is employing a combination of theoretical development of 
the techniques, Monte Carlo simulation to check the performance of the 
techniques under controlled conditions, and validation with actual response 
data. 
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Progress: Initial research focused on efficient procedures for estimating the 
marginal posterior distributions of individual ability parameters. Approximations 
by Leonard, bv Tlemey and Kadane, and by Lindley were studied, but in the 
general case were not practical for routine use. Research examined a new 
Bayesian approach based upon normal approximation of the posterior item-parameter 
distribution and its theoretical properties are under study. 

Potential Applications ; The use of model-based procedures for test 
development and scoring in Navy and Marine Corps training schools has been 
hampered by statistical estimation problems. Standard procedures for 
estimating IRFs require large item pools and substantia] samples of 
examinees— requirements which are virtually unattainable in the typical 
military training context. When standard procedures are used with small item 
pools and with small numbers of examinees, results can be misleading. This 
effort will improve that situation. 



NON-PARAMETRIC ESTIMATION OF THE LATENT-TRAIT DENSITY 
NR 150-537 

University of Missouri/Speckman 

Sciuentific Obiective; This research is exploring two new approaches for 
characterizing ability densities underlying performance on psychological 
tests. Each approach represents the ability aensity as an exponential spline 
in which the exponent is in a finite-dimensional space of linear spline 
functions. One approach is seeking to maximize the marginal likelihood over 
than class. A second approach is seeking to maximize a penalized marginal 
likelihood. Each approach is expected to provide smooth non-negative density 
estimates without further constraints. 

Approach; Initial work is focused on theoretical and numerical developm<int of 
the techniques and on evaluation via Monte Carlo simulation. Subsequent work 
will evaluate performance using actual response data via suitable 
cross-validated designs. Ultimately, these techniques will be folded into 
marginal-maximum-llkelihood approaches for estimating item-response functions 
(see NR 150-535) 

Progress; This work began late in FY85, and no Status Reports have yet been 



Potential Applications ; The accurate characterization of unobservable ability 
densities is a central problem in psychological measurement. It is a central 
part of the assessment of model appropriateness (NR 150-518), of the 
non-parametric characterization of item-response functions (NR 150-467 & 
NR 150-518), and of efficient estimation of individual abilities. This work is 
expected to substantially increase the accuracy of ability density estimates 
and thereby increase the fidelity of other measurement results. 
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THEORY OF ROBUSTNESS FOR ITEM-RESPONSE THEORY 
NR 150-522 

Advanced Statistical Technologies Corporation/Jones 

Scientific Obiective ; This work is seeking a theoretical framework and 
tractable procedures within that framework for gauging the robustness of 
estimators of item-response functions and of examinee ability. In addition a 
theory of optimal estimation is being formulated and, where feasible, optimal 
estimators derived. 

Approach ; (1) Influence functions appropriate for item-response theory are 
explored; (2) Formulas for the efficiency of several robust estimators 
are derived and empirically studied; and (3) Several item-response- function 
models are studied and rigorous characterizations of neighborhoods of these 
- models re derived as a basis for examining optimality. 

Progress ; The sensitivity of maximum-likelihood procedures to unusual data 
and to model misspecif ication has been characterized. Work on the sensitivity 
of maximum-likelihood procedures to unusual data has extended Pregibons' methods 
to item-response theory; to accomplish this, the Newton-Raphson algorithm (the 
standard numerical technique for obtaining maximum-likelihood estimates) was 
reformulated in terms of non-linear least squares. This reformulation 
permitted the derivation of closed formulas for the effects of deleting one or 
two observations from a data set on estimates, confidence intervals, and 
goodness-of-fit statistics. Work aimed at gauging the sensitivity of maximum- 
likelihood procedures to mode] misspecif ication obtained a general parametric 
class of item-response functions within which one can characterize neighborhoods 
of the standard 1-, 2-, and 3-parameter-logistic models. Examination of the 
breakdown points of the standard models indicated that at low abilities the 
bias of the maximum-likelihood ability estimator for the 1- and 2-parameter 
models was extremely sensitive to model misspecif ication. 

Potential Applications ; In practical applications of item-response theory it 
is guaranteed that the assumed model and the true model will differ. 
Consequently, understanding the sensitivity of standard estimation techniques 
to :,uch differences is essential. In additr'on the maximum-likelihood 
diagnostics under development here identify items and people whose behavior 
seems to depart markedly from model predictions. These can be used to adjust 
models, to exclude aberrant items, and to identify examinees who may have 
exceptional knowledge. 



HIERARCHICAT.-BAYES ESTIMATION OF ITEM-RESPONSE MODELS 
NR 150-539 

Educational Testing Service /Mis levy 



Scientific Objectives ; Techniques for estimating item-response functions 
perform poorly when sample sizes or test lengths are small. This work will 
improve that situation by extending hierarchical Bayesian techniques to this 
context. Essentially, techniques are developed which take advantage of 
collateral information about persons (e.g., years of schooling) and items 
(e.g., cognitive processing requirements) to improve estimation within the 
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item-response-theory framework. 



Approach; The hlerarchical-Bayeslan approach is employed to estimate 
the relationships between model parameters (i.e., parameters describing 
attributes of individual items and examinees) and collateral information. 
Bayesian and empirical-Bayesian estimation procedures provide modal 
estimates of item and structural parameters via a variant of the EM algorithm. 
Results will be evaluated on data from the Armed Services Vocational Aptitude 
Battery (ASVAB) and from the National Assessment of Educational Progress 
(NAEP) program. 

Piogress: A theory for incorporating collateral information about examinees 
through a generalized linear model was developed. The case of categorical 
collateral variables (e.g., high school graduation and gender) was 
completed. Empirical results with ASVAB data indicated that use of these 
procedures increased the precision of item parameter estimates substantially. 
A more general model incorporating collateral information about both items 
and examinees is under development. 

Potential Applications; On-line calibration of new test items is currently 
hampered by the sparsity of data at the level of Individual examinees—in 
adaptive administration of ASVAB only 15 responses to previously calibrated 
items are planned. Judicious use of auxiliary information about examinees and 
items could increase the precision of item parameter estimation and reduce its 
bias by amounts comparable to increasing the test length by 20%, without 
requiring additional testing time. 



EQUATING COMPUTER-BASED-ADAPTIVE AND PAPER-AND-PENCIL ASVAB 
NR 150-534 

Johns Hopkins University/Green 



Scientific Objectives; This work is developing techniques for equating 
computer-based-adaptive and paper-and-pencil versions of the Armed Services 
Vocational Aptitude Battery This includes the development of techniques for 
verifying and re-estimating item-response functions with adaptive data sets 
and for equating multi-faceted paper-and-pencil and adaptive tests. 

Approach ; (1) Theoretical expressions for the attenuation in item parameter 
estimates as a function of ability distribution and sample size are developed. 
(2) Issues concerning the design of the recalibration/equating data collection 
study are examined via Monte Carlo simulation. (3) Techniques for detecting 
mode-of-administration and mode-by-item interactions are developed and 
validated with simulation data. (4) A variety of techniques for equating 
adaptive scores to number-correct scores on the Auto-Shop subtest are compared. 

^y°g^^^^ = A detailed study of the effects of computer presentation and of 
alternate scoring methods for ASVAB speeded subtests has been completed. 
Results indicated that a scoring method gauging the number of correct responses 
per minute was superior to the traditional method of counting the number of 
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correct answers in a fixed time period. The reliabilities of scores on the 
computer-presented versions were at least as high as that of the paper-and-penc il 
versions. Examinees responded faster on the computer-presented versions than 
on their paper-and-pencil counterparts. Correlations between computer-based 
and paper-and-pencil versions of the numerical-operations subtest were 
substantial. 

In other work investigators examined the viability of a number of options for 
DoD's adaptive ASVAB: (a) Simulations of data collection strategies showed 
that data from 5 "seeded" items will not be sufficient for item re-calibration; 
data from adaptlvely-administered items may be required. (b) Simulations of 
the impact of fixing lower asymptote parameters at incorrect values showed 
that estimates of other parameters compensate for the incorrect lower 
asymptotes. The net result is that ability estimates are affected very little, 
(c) Simulations of the impact of reduced pool size confirmed that the 
information loss is modest, (d) Simulations are currently underway to 
examine the effects of exposure-control strategies on the performance and 
quality of adaptive tests. 

Potential Applications ; To get off the ground with a computer-based adaptive 
ASVAB, the Services need (a) large banks of test questions on which item 
response functions have been calibrated and (b) mathematical functions which 
map score55 on the computer-based adaptive test onto scores on the equivalent 
paper-and-pencil version of the test. Since initial calibrations were 
necessarily obtained from data gathered in paper-and-pencil format and 
examinees were told that the test was experimental, the generalizability of 
the item-response functions to the operational environment is uncertain. 
Furthermore, in the ASVAB environment decisions about about a candidate's 
suitability for a particular Service School must be made on the spot. 
Therefore, techniques for bootstrapping are needed. At the least this work 
will make clear how difficult accuracy really is. It should also make clear 
the extent to which we can have confidence in item-response functions 
calibrated under experimental instructions and in paper-and-pencil format. 
At best this work will provide techniques for equating multi-faceted tests 
by adjusting calibrations. 

Funding : NPEDC 



BAYESIAN INFERENCE IN FACTOR ANALYSIS 
NR 150-536 

Sloan-Kettering Cancer Research Center/Wong 

Scientific Objectives ; This effort extends previous work on Bayesian 
approaches to factor analysis. A framework for characterizing the moments of 
the marginal posterior distributions of the factor loadings, factor scores, 
and uniqueness variances is developed; expressions of estimation errors 
are derived and compared with those obtained under maximum-likelihood 
and Rayes- joint-modal approaches; and the sensitivity of the marginal moments 
to variations in the prior distributions is examined. In addition, these 
marginal procedures are extended to che study of oblique factor models, of 
models with a priori zero factor loadings, and simultaneous factor models. 

Ill 



Approach; This work employs a combination of theoretical development of 
the techniques and Monte Carlo simulation to check the performance of the 
techniques under controlled conditions. A central element of this research 
involves approximating the marginal moments of the posterior distributions of 
factor loadings, factor scores, and uniqueness variances. Techniques based 
upon an extension of the so-called Tiao-Zf liner expansion and refinements of 
the EM algorithm are being explored. 

Progress: This work unit began late in FY85, and no Status Reports have yet 
been received. 

Potential Applications; Technical difficulties with maximum-likelihood and 
previous Bayesian approaches to factor analysis of ability data include- (a) 
flat likelihood surfaces in a neighborhood of the global maximum; (b) multiple 
local maxima; (c) asymptotic sampling theory which is inadequate even for 
moderate samples, and small-sample theory which is nonexistent; (d) dogmatic 
treatment of loadings judged on a priori evidence to be small; (e) under- 
estimation of uniqueness variances; and (f) lack of analyses gauging the 
sensitivity of these procedures to perturbations in the prior distributions. 
This research will ameliorate many of those problems. 



BAYESIAN MODELS FOR PERSONNEL SELECTION AND CLASSIFICATION 
NR 150-521 

University of lowa/Novick 

Scientific Obiectives; In previous work Novick developed sophisticated 
methods for fitting regressions simuxtaneously in multiple groups, for drawing 
inferences from non-experimental data sets, and for setting cut scores for 
psychological tests in decision-making environments. The validity and 
usefulness of those techniques is examined further by applying them to 
prototypical military data sets. 

Approach; The relationships between Armed Services Vocational Aptitude 
Battery (ASVAB) scores and biographical and demographic variables on the one 
hand and military performance variables on the other are being examined using 
Bayesian simultaneous regression techniques within schools/locations/Services 
and occupational groupings. Formal Bayesian utility assessment and maximization 
techniques are being employed to determine optimal cut scores for military 
selection and for assignment to selected schools and to selected military 
occupations. 

Progress; Completed research provided evidence for differential prediction 
by gender, training course, an educational background, but not by race. 
One study showed that the use of educational background as a moderator 
substantially reduced differential prediction on ASVAB variables. In another 
set of studies the use of simultaneous estimation of regressions showed that 
courses within broad categories, such as mechanical or clerical, can often 
be clustered to provide subgroups in which common prediction is warranted. 
Several methodological developments were obtained. An improved method of 
Bayesian estimation in two-by-tT.-o contingency tables and an improved Bayesian 
regression algorithm were developed. 
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Potential Applications : With present selection technology, prediction 
functions for low-frequency schools/occupations or for low-frequency 
population subgroups are poorly estimated and cut scores are established 
without resort to utility considerations. This work will Improve that 
situation. 

Funding ; OASD (M&RA) and MEPCOM 
Reports ; 

Dunbar, S. B., Mayekawa, S., & Novick, M. R- (1985). Simultaneous estimation 
of regression functions for Marine Corps technical t raining specialties 
(Technical Report 85-1). Iowa City, LA: University of Iowa, Center for 
Measurement. (AD 150477) 

Dunbar, S. B. & Novick, M. R. (1985). O n predicti ng success In training for 
males and females ; Marine Corps clerical specialtie s and ASVAB forms o and 7 
(Technical Report 85-2) . Iowa City, lA; University of Iowa, Llndqulst Center 
for Measurement. (AD 152080) 

Leonard, T., & Novick, M. R. (1985). Bayeslan In ference and diagnostics for 
Lhe three parameter logistic model (Technical Report 85-5). Iowa City, lA: 
University of Iowa Llndqulst Center for Measurement. 

Leonard, T. , & Novick, M. R. (1985). Bayeslan full rank margl nallzatlon for 
t wo-way contingency tables (Technical Report 85-4). Iowa City, lA: University 
Iowa Llndqulst Center for Measurement. (AD 153841) 

Mayekawa, S. (1985). Bayeslan factor analysis (Technical Report 85-3). Iowa 
City, lA: University of Iowa Llndqulst Center for Measurement. (AD 154184) 

****** 

THE FOLLOWING WORK UNITS WERE COMPLETED DURING THE PAST YEAR 

MODELS FOR MULTIDIMENSIONAL TESTS AND FOR HIERARCHICALLY-STRUCTURED TRAINING 

MATERIALS 

NR 150-474 

American College Testing Programs/Reckase 

Scientific Oblectlves : The objectives were to develop and to evaluate 
probabilistic models: (a) for the performance of Individuals on test Items 
which require proficiency on more than one skill and (b) for the relationship 
between performance on two units of Instruction when one contains material 
which is prerequisite to the other. 

Approach : The approach taken was to logically evaluate the characteristics 
of potential models, to develop estimation procedures for model parameters, 
and to evaluate the models on their ability to describe real test data. 
These steps were performed separately for a wide class of models. 
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Results and Conclusllons! Work on multidimensional models extensively studied 
a multidimensional extension of the two-parameter-logistic model. Estimation 
procedures were developed and the results were validated using simulated and 
real test data. A conceptual framework was outlined for the interpretation of 
model parameters and multidimensional extensions of the unidimensinnal notions 
of item difficulty, discrimination, and information were obtained. Remaining 
deficiencies that were noted include the following: (a) estimation programs 
required an excessive amount of computer time when more than two dimensions 
were specified, and (b) Procedures were needed to assess the fit of the model. 

In other work a model of the relationship between performance on instructional 
modules which are hierarchically arranged was developed and evaluated using 
simulated and real test dat.. One problem with the use of the model surfaced, 
however. To accurately estimate model parameters, in a calibration study 
trainees must be advanced to higher-level units even when they have performed 
poorly on lower-level units. Because this is viewed as poor educational 
practice, needed data often cannot be obtained. 



Reports ; 

Reckase. M. D. (1985). Models for multidimensional tests and hie rarchically 
structured training materials (Final Rppnrf nwR-«-;-n Iowa City. lA- The 
American College Testing Program. 



THE USE OF TAILORED TESTING IN INSTRUCTIONAL PROGRAMS 
NR 150-499 

American College Testing Programs/Reckase 

Scientific Objective; This project developed. Implemented and studied a 
computer-based testing system for the Navy's Radar Technicians Course at Great 
Lakes Navy Training Center. The objective was to produce a more efficient 
testing procedure and to Improve the quality of measurement. 

Approach; The system was a multi-level, microprocessor-based computer network 
with each testing station capable of operating as a stand-alone test 
administration system. The system was used to administer tests in a 
sequential, fixed-length format. Although software was also developed for 
adaptive administration, the system was never used for adaptive testing 
because of repeated hardware failures. 

Results and Conclusions; In addition to system development and implementation 
work, research studies were conducted on model selection study, mode-effects, 
and an attitude survey. The model-selection study compared ability estimates 
obtained using the Rasch model with ability estimates obtained using the three- 
parameter-logistic (3PL) model. Results indicated that adaptive ability 
estimates from the Rasch and 3PL models are highly correlated; there were no 
systematic differences in measurement quality between them. Consequently, 
since the Rasch model procedure is substantially less expensive, it was 
recommended for sparse data environments like those for adaptive testing. 
The mode-effects study compared performance on paper-and-pencil and 
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computer-based versions of the 86 items used in the Navy's Radar Technicians 
Course. Analyses of estimated item difficulty parameters identified 26 items 
(or 30%) affected by mode of administratlcr; 11 items were harder when 
administered by computer, and 15 were easier. Several hypotheses such as 
differences being related to the airount of verbal material in the item 
or item format differences were eliminated; however, no obvious explanations 
were identified. 

Results of the attitude survey indicated that electronics-technician trainees 
had little difficulty with the computer-based system. Trainees reported 
difficulty in reading the screen, with eye fatigue, or in finding the proper 
keys on the keyboard. The only commonly voiced complaint concerned the lack 
of a procedure for reviewing previously-answered items. 

Potent ial Applications ; This project demonstrated tuat microcomputers could 
be effectively used to administer achievement tests in a Navy instructional 
environment. Second, it showed that the Rasch model may be adequate for 
adaptive testing in the typical data-sparse Navy and Marine Corps training 
environments. Third, it underscored the need for caution in adapting paper- 
and-pencil test items to computer format. Fourth, it suggested the need for 
procedures which can accommodate review and answer-changing in the course of 
adaptivv tests. 



MARGINAL-MAXIMUM-LIKELIHOOD ESTIMATION IN ITEM-RESPONSE THEORY 
NR 475-018 

National Opinion Research Center/Bock 

Scientific Objectives ; In this work, the usefulness of recent theoretical 
developments in marginal-maximum-likelihood methods for estimating item- 
respouse functions, for gauging the dimensionality of tests, for simultaneous 
multidimensional adaptive testing, and for estimating the trait distrxoutions 
of selected subpopulations were explored in a military context. 

Approach; Empirical studies employing Armed Services Vocational Aptitude 
Batt7r~(ASVAB) data from the Profile of American Youth program we-e used, 
cross-validated. Estimated item-information functions were examined in 
light of item content and evidence for multidimensionality . Several methods 
for estimating subtest trait distributions were also examined and compared. 

Re sults and Conclusions ; (1) A four-phase Hayes constrained estimation 
procedure extended Bock's earlier work to the three-par-meter-logistic 
situation. These procedures were successfully applied to the Profile of 
American Yuuth data set. (2) Analysis of the dimensionality issue suggested 
that the Auto-Shop subtest was two-dimensional. (3) A multiple-category model 
for iucorpnrating information froir wrong responses to multiple-choice items 
was dev. loped and tried with the ASVAB data set. 

Potenti al Applications ; (a) The multiple category model makes possible the 
recovery of Information available in wrong responses to multiple-choice test 
items and can thus increase the speed and accuracy of computerized adaptive 
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testing. The model also provides so-called category operating 
characteristics. Those functions can be used by test designers to assess the 
effectiveness of distractors in multiple-choice test items; and (b) The full- 
information factor analysis developed in this work ought ^o replace traditional 
linear factor analysis. The full-Information approach not only conforms more 
closely to the latent-trait non-linear notion of dimensionality, but also 
reduces the likelihood of obtaining spurious guessing factors. 

Funding ; ONR Manpower R&D Program. 

Reports ; 

Bock, R. D., Gibbons, R., & Muraki, E (1985). Full-information item factor 
^^^^ysis (Technical Report 85-1). Chicago, XL: University of Chicago National 
Opinion Research Center. 

Mislevy, R. J., & Bock, R. D. a984). Item operating characteristics of the 
Armed Services Vocatio nal Aptitude Battery (ASVAB) . f orm 8A (Technical Report). 
Chicago, XL: University of Chicago National Opinion Research Center. 

Thissen, D,. & Steinberg, L. (1984). A response model for multiple choice 
items (Technical Report). Chicago, XL: University of Chicago National 
Opinion Research Center. 

Archival Publications : 

A L 

Mislevy, R. J. (1984). Estimating latent distributions. Psychometrika . 49, 
359—381. 



Thissen, D. b Steinberg, L. (1984). 
items. Psychometrika . 49 , 501-519. 



A response model for multiple choice 



ADAPTIVE ASSESSMENT OF SPATXAL ABXLXTXES 
NR 154-531 

Educational Testing Service/Bejar 



Scientific Objectives ; This research developed and evaluated latent-trait 
techniques for modelling individual differences in performance on spatial 
tasks. 

Procedure: Investigators conducted a comprehensive review of the literature 
to select a set of spatial tasks for study. Appropriate latent-trait models 
and estimation procedures were developed. Performance data we.e collected 
and models calibrated. Finally, results were linked to explanations of 
individual differences obtained through other approaches. 

Results and Conclusions; A three-dimensional spatial-rotation task and a 

hidden-figures task were selected for study and response data were collected 

on approximately 200 participants, Samejima's continuous response model (see NR 
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150-467) was modified to account for the fact that examinees cannot respond 
instantaneously and that soTne examinees may run out of time. 

Data for the rotation task were collected from the 200 examinees each 
responding to 80 items. There were eight basic items presented at five angles 
in their "true" and "false" versions. Models were fit to the response-time 
data for the "same" and "different" responses separately. Analyses examined 
the relationship between the size of the difficulty parameter and certain item 
attributes. For the three-dimensional spatial-rotation tasks, results 
confirmed the well-established linear relationship between response time and 
rotation angle on the "same" items. Analyses provided no new insights into 
response-time differences on "different" items. Analyses of the 
hidden-figures task showed that an item-generation algorithm can be formulated 
to produce item "clones" with similar psychometric characteristics. 

Potential Applications ; It was anticipated that the application of these 
mathematically-sophisticated modelling techniques would filter out much of the 
"noise" in the response data and thereby increase measurement fidelity and our 
understanding of the nature of the individual differences on these tasks. 
Since many Navy and Marine Corps tasks engage spatial capabilities, it was 
anticipated that this increased understanding would suggest improvements in 
selection, task design, and training. 

Reports ; 

Bejar, I. (1986). Adaptive assessment of spatial abilities (Final Report) 
Princeton, NJ; Educational Testing Service. 

Bejar, I. (1986). The Psychometrics of mental rotation (Technical Report 
RR-86-19) . Princeton, NJ; Educational Testing Service. 

Bejar, I. (1986). Analysis and generation of hidden figure items: A cognitive 
approach to psychometric modelling (Technical Report PR-86-20) Princeton, 
NJ; Educational Testing Service. 

Bejar, I. (1986). Adaptive assessment of spatial abilities (Final Report) 
Princeton, NJ; Educational Testing Service. 

Archival Publications: 

Bejar, I. (1985). Educational diagnostic assessment. Journal o f Educational 
Measurement, 21, 175-190. 
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K. CHARACTERISTICS OF THK HUMAN LEARNER 

This facet of the research program in psychology aims to understand the 
human system that are trying to train to function effectively in Naval 
jobs. What resources does the human learner have available that can be worked 
with? There are three primary questions about the characteristics of this 
system: 

* How is the information that is presented coded and stored for future 
use? 

* What processes are available within the system to operate on that 
information? 

* What are the mechanisms of control that govern the selection and 
operation of those processes? 

These questions are generally recognized as important in experimental 
psychology, but there is a particular emphasis on developing those aspects of 
psychological theory that will help us to understand how the human system 
acquires and performs complex skills. A central challenge in this research is 
to factor out invariant structural characteristics of the human information 
processing system from modifications that learning builds upon those 
characteristics. In this way, we can achieve an understanding of sound bases 
for personnel selection and an understanding of the way in which 
desirable modifications of human abilities can be brought about. Research on 
the storage of information in short-term memory is suggesting how we can 
create new structural resources in the human system in order to enhance the 
ability to deal with complex and information-intensive tasks. 

Much of the work in this cluster has focused on the spqtial abilities 
that figure in many Navy jobs (e.g., aircraft control, surface operations, 
electronic maintenance) . The approach contrasts with the traditional 
psychometric approach, instead aiming at a fine-grained understanding of 
available information processing operations. In addition to fine-grained 
analysis of individual differences, this research has aimed to determine the 
degree to which specific components of ability are trainable. This research 
should be valuable In selection and training and for the design of diagrams, 
job aids, etc., that will meet the needs of a wider range of personnel. 

This research focuses on attentional allocation and capacity, with a view 
to understanding the abilities required to operate complex, high demand 
systems such as aircraft. The way in which appropriate training can reduce 
the effective attentional demands of complex tasks has been a major theme, 
which is already influencing more applied training research. 

Work in this cluster is corducted with and passed to applied interests 
through both formal and informal conferences. Representatives of Navy 
laboratories, of other-service laboratories, and both military and civilian 
R&D managers are invited to these meetings. Meetings discussing the research 
of this cluster have been scheduled at Yale-Haskins Laboratory in March 1984, 
Harvard University in October 1985 and the University of Oregon in October, 
1986. 
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COMPREHENSION PROCESSES IN MECHANICAL REASONING 
NR 150-476 

Carnegie-Mellon University/ Just 6 Carpenter 

Scientific Objectives ; The major goal of the research is to provide an 
explanation of how people come to understand the workings of a mechanical 
device or machine by reading a technical text and inspecting an accompanying 
diagram. 

Approach ; This research will use a combination of techniques from 
expciimenta] psychology and computer simulation. The experimental methodology 
involves the tracking of observers' eye fixations as they read technical texts 
and solve visual problems. The record of their points of regard in the 
display provides an index of the sequence and durations of the ongoing mental 
processes. Other behavioral measures collected are reaction times and 
patterns of errors in tests of mastery of the device. The comp \ter simulation 
work involves the construction of production systems that perform the visual 
and mechanical tasks similarly to the humans, preserving the sequential and 
temporal patterns revealed by the eye fixations. The production systems will 
be written under CAPS, a parallel, activation-based, production system. 

Progress ; This project was begun late in FY-85, and no Status Reports have 
yet been received. 

Potential Applications ; The research may lead to new instructional 
technologies, involving gaze-contingent text and diagram displays, and 
an^ lations of mechanical systems. The intent of these innovations is to 
optimize the match between the displayed information and the reader's state of 
thought at any given instant. In particular, this approach might be capable 
of instantaneously detecting "mental logjams" and assisting in their breakup. 

Archival Publications ; 

Just. M.A., & Carpenter, P. A. (1985). Cognitive coordinate systems; Accounts 
of mental rotation and individual differences in spatial ability. 
Psychological Review , 92, 137-172. 



TRANSFORMATIONS OF SKORT-TERM VISUAL MEMORY 
NR 15/f-533 

University of Pennsylvania/Sternberg 

Scientific Objectives ; To investigate how visual information is represented 
in memory during the first few seconds after a display. Issues include how 
many representations must be postulated, when they are availabTa, their 
properties, their durations, the transformation from one to another, and 
effect*^ of the class of display elements. Unlike much previous research on 
these issues, this work employs arrays small enough so as not to overload 
memory; the aim is to understand performance under conditions of high 
accuracy. 
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Approach; By applying time pressure to the observer under these conditions we 
induce mechanisms to reveal themselves, not by how they fail, but by how much 
time they need to succeed. Thus, measures are taken of how much time a person 
needs to retrieve Information accurately from memory of the display, and how 
this depends on time since display, number of displayed elements (e.g., 
alphanumeric characters, nameless shapes), information required, and probe 
type. A critical feature of the approach is he use of a range of 
d-values. (This feature seems essential since earlier work found that 
whether retrieval requires more or less time as the probe is delayed 
depends on d) . 

Progress; The research ha- developed and tested four diverse experimental 
paradigms that express the approach. All four produced orderly data 
that revealed sharp changes in the pattern of retrieval times — and hence 
in the internal representation — within about one second of the display. 
Among the more surprising findings were inverse effects of delay on search 
rates and reciting speed, rapid loss of direct access by location, functional 
equivalence of visual and tactile location markers, similarity of memory 
transformations for familiar symbols and unfamiliar nameless shapes, and an 
advantage of array over sequence even after several seconds. 

Potential Applications: Knowledge of how visual information is assimilated is 
important in the design of displays and tasks. Particular findings with 
practical implications include effectiveness of tactile stimuli in directing 
visual attention, advantage of location over identity probes for large arrays, 
time costs of increasing array size for different tasks, and the advantage of 
presenting visual information arrayed spatially rather than sequentially for 
accurate, fast, and flexible use. 



COPING WITH NOVELTY AND HUMAN INTELLIGENCE 
NR 154-534 

Yale University/Sternberg 



Scientific Objectives ; The goal of this research effort is to isolate the 
parameters of problem-solving performance that measure the ability to deal 
with novelty, particularly the ability to deal with relevant novelty in 
counter-factual (hypothetical) reasoning. 

Approach: Converging operations are used in three domains and in multiple 
t^sks within each domain to establish the construct validity of the 
operationalizations of the ability to cope with novelty in counterf actual 
reasoning. Internal and external validation are used to ensure both the 
satisfactory isolation of the relevant parameters, and the convergent and 
discriminant validity of the parameters with respect to established 
psychometric measures. By isolating the parameters that bear specifically on 
the ability to deal with novel problems, it will be possible to integrate a 
theory of this ability with existing ability theories derived from 
psychometric and cognitive research. 

Progress: Four series cf experiments have been completed and support 
in different ways the validity of an ability to cope with novelty in 
counter-factual reasoning. The experiments have elaborated the 
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information-processing bases of this ability pointing out that it is 
not a unitary one. Rather, the ability involves component siibskills 
requiring both comprehension of novelty and reasoning with the information 
comprehended. 

Potential Applications ; This work will contribute to the basic research 
foundations for personnel selection. In particular, it should contribute to 
our ability to select for individuals who can cope with the demands of novel 
situations. 



INTELLIGENT MAINTENANCE TRAINING SYSTEM 
NR 535-001 

University of Southern Calif ornia/Towne 

Scientific Objectives ; To develop an experimerftaJ training system for 
intelligent computer-assisted maintenance training that includes both (1) 
computer-aided methods for analyzing the practice requirements of maintenance 
training and (2) a general training system that will provide individualized 
selection and sequencing of exercises and an automated tutor that will 
coach students during exercises. This system will incorporate the Navy' s 
existing General Maintenance Training Simulator and will be implemented for 
blade fold rotor brake (helicopter) maintenance training. 

Approach ; Relations among training exercises and required skills will be 
represented in a database, constructed with computer-aided analyses. 
Artificial intelligence techniques will be used to model the acquisition of 
student skills, to select appropriate exercises, to analyze ongoing student 
performance, and to provide appropriate tutorial interventions. 

Progress ; A detailed planning study was completed. Internal data structures 
for the system have been specified. Editors to configure the system for 
particular applications were completed from four areas: (1) gererlc elements 
(replaceable units, indicators, and controls); (2) graphic system 
configuration; (3) student knowledge and skills; and (A) PROFILE 
(diagnostic expert system) data. 

Potential Applications ; It is anticipated that this system will allow an 
experienced maintenance technician to easily enter system-specific information 
to produce both training simulation and an automated intelligent tutor for any 
of a wide range of electrical, mechanical and hydraulic systems. 

Reports ; 

Towne, D. M., Munro, A., Pizzinig, A., Sumon, D., and Johnson, W. B., (1985) 
Development of Intelligent Maintenance Training Technology ; Design Study . 
Los Angeles, LA: University of Southern California, May, 1985. 

Funding ; NPRDC 
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INTERACTIVE ACTIVATION MODELS OF SPEECH PERCEPTION 
NR 667-542 

University of California, San Dlego/Elman & McClelland 

Scientific Objectives; Development of a model of human speech perception that 
is psychologically accurate In understanding the details of human speech 
perception experiments, and that is computationally adequate to recognize 
words and phonemes in real speech. 

Approach; Interactive activation models are used as a theoretical framework 
wirhin which to build models of speech perception. Interactive activation 
models are those in which processing is carried out by a large number of 
highly interconnected processing elements. A model of speech perception is 
developed in this fi^mework and then applied to an analysis of a number of 
different issues in speech perception. 

Progress; The interactive activation model of speech perception (TRACE) 
developed during the previous year's contract work has been extended by (a) 
addressing, by experimental means, the questions of how modular the speech 
perceptual apparatus is; and (b) developing a new technique for applying a 
"connectionist" solution to the problem of how human speakers (or machine 
analogs) might learn the knowledge structures necessary for recognizing 
speech. 

Potential Applications ; The basic interactive activation is potentially 
applicable to a wide range of Navy-relevant human behavior which has resisted 
analysis in terms of standard information processing approaches, especially in 
areas such as perception, automated response generation and decision-making 
under time pressure, etc. More specif icrlly, the TRACE model of speech 
perception is a step toward the development both of an understanding of what 
is necessary for effective speech communication, thus aiding design of 
speech-production machinery, and an understanding of the basic principles at 
play in perception, thus aiding in the design of speech perception machinery. 



FUNCTIONAL SYNERGIES IN VOLUNTARY MOVEMENT AND SPEECH 
NR 667-509 

Haskins Laboratories/Kelso, Saltzman, & Turvey 

Scientifi c Objectives ; The current project seeks to understand how the 
multiple degrees of freedom of the skeletomuscular system are organized to 
produce coordinated movement. Integrated with our experimental work involving 
discrete and rhythmical tasks is the further development of mathematical 
descriptions of such functional units and a description of their physical 
basis. 

Approach; Kinematics (e.g., velocity), kinetics (e.g., torque), and related 
muscular activity are being measured In tasks that involve movement of (1) 
limbs, hands, and fingers, both singly and in concert and (2) speech 
articulators. Key experimental strategies: (a) apply perturbations to ongoing 
movements to identify functionally and remotely linked articulators and (b) 
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systematically vary kinematic and kinetic parameters (e.g., frequency and 
inertia) to identify properties that renain invariant across transformation. 
Mathematical (task dynamical) models are being used to describe trajectories 
and compensatory behavior arising from both strategies, while a law-based 
account under development is used to rationalize invariant properties emerging 
from strategy (b) . 

Progress : The results of a series of studies provide strong evidence that the 
various muscles paruicipating in an action function as a flexible unit to 
achieve a functional goal, despite external perturbations that disrupt the 
normal course of actions. There appear to be only a few stable phase 
relations in which the motions of two limbs can be combined. Properties of 
motion transitions have been shown to resemble properties of non-equilibrium 
phase transitions of chemical substances. A mathematical model has been 
developed to predict the natural rate of the combination of two motions with 
different natural rates. 

Potential Applications ; Knowledge of how functional units are formed and 
dissolved should provide a rational basis for understanding processes of skill 
acquisition (developmentally or in adults), skill loss (due to Injury or 
aging), and the production of speech. 

Archival Publications ; 

Kelso, J. A. S., V. Bateson, E., Saltzman, E. L. & Kay, B. (1985). A 
qualitative dynamic analysis of reiterant speech production; ?has^ portraits, 
kinematics, and dynamic modeling. Journal of the Acoustical Society of America , 
77, 266-280. 

Kelso, J. A. S. b Tuller, B. (1985). Intrinsic time in speech production. 
Journal of the Acoustical Society of America , 77^ (Suppl. 1), S53. 

Saltzman, E. L. (1985). On the coordination and regulation of complex 
sensorimotor systems (A review of H. £. A. Whiting (Ed.), Human motor 
actions; Bernstein reassessed). Contemporary Psychology , 30 , 531-533. 

Turvey, M. T., & Carello, C. 0985). The equation of information and meaning 
from the perspectives of situation semantics and Gibson's ecological realism. 
Linguistics and Philosophy . 8^, 195-202. 



LEARNING AND COORDINATION OF COMPLEX PSYCHOMOTOR SKILLS 

NR 667-510 

The Technion/Gopher 

The University of Illinois/Donchin & Karis 

Scientific Objectives ; The aim is the study of the mechanisms and processes 
involved in the acquisition and operation of complex psychomotor skills. 

Appraoch ; A data-entry skill on a newly designed, two-hand chord keyboard was 
used as a model task. The keyboard contained two separate 5-key panels, each 
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capable of producing the full set of letters. The performer was free to 
develop strategies of coordination, and in principle was capable of typing two 
different texts simultaneously. The system provided an interesting testbed 
for fundamental issues in the study of complex skills. Prototypes exist both 
at the Technion and at the University of Illinois, and experiments have been 
conducted in both the Hebrew and English languages. 

Progress; Experiments were conducted with two types of tasks: ente-ing 
sequences of single letters by one hand and pairs of letters by two 

n^'^c,:.??'^/^^^"* °' The main conclusions were: 

U; Skilled movements are represented by general schemas that were separate 
from the direct parameters of the actual movement; (2) Schemas varied In 
their formatting elements, depending upon the coding principles that were 
used to encode them; (3) One store of representations served both hands; (4) 
The store had spatial composites and was localized in the right hemisphere; 
and (5) Responses to letter pairs that had to be typed simultaneously were 
searched and prepared for sequentially. A four-stage process of transcription 
emerged from these results. These were: letter Identification, code retrieval, 
conversion to hand format, and execution of movement. Factors influencing 
processing efficiency were revealed both within and across stages. 

Potential Applications; The results of this research suggest new 
possibilities for the design of data-entry and other kinds of manually 
operated controls. They also Indicate the effectiveness of novel training 
procedures, using imagery, to help trainees learn to operate such devices 
quickly and efficiently. 

Reports ; 

Gopher, D, Koenig, W. Karis, D, Donchin, E. (1984). An Information pr ocessing 

approach to die study of data entry skill..,; The effects of representat l^ 

rules and coordinat ion requirements (Technical Report CPL84-14). Urbana, IL; 
University of Illinois, Department of Psychology, October 1984. (AD 150295)* 



ATTENTION AND PERFORMANCE 
NR 667-523 

University of Oregon/Posner 
Washington Unlverslty/Keele 



Scientific Oblectlves: The objective is to test a theory which specifies the 
relationships between neural systems and elementary Information-processing 
operations. The operations of Interest underlie the control of visual 
attention, language comprehension, and the control of movement. 

Approach; The research Is using techniques for the analysis of the time 
course of attention control and movement coordination. These techniques 
produce measures of the speed and accuracy with which the subskllls of 
attention and motor control are performed In a given individual. Measurements 
of these subskllls are made for normal young adults and for various populations 
of individuals with brain lesions at particular sites. The resulting 
performance comparisons are being used to test a theory of attention and 
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motor control which assigns particular information-processing activities 
to particular brain sites. 

P rogress : Research with normal and brain-damaged patients was carried 
out to identify the neural bases for che operations underlying the movement of 
attention in visual space. These studies revealed that (1) Unilateral 
lesions of the posterior parietal lobe produce a .-.pacific disruption of the 
ability to disengage attention from a prior spatial location in order to move 
it to new attention-demanding sites located in the contralateral visual field; 
(2) Lesions in the midbrain region affect the ability to move attention from 
its current fixation; (3) The movement of attention from a spatial fixation 
produces a reduced ability to immediately return attention to that fixation 
site; and (4) The neural mechanisms underlying movement of attention in visual 
space are distinct from, and hierarchically below, the more general mechanism 
of attention control that determines the modality to which attention is 
directed. 

Potential Applications ; This research seeks to identify and characterize 
certain fundamental information-processing operations underlying many Navy 
jobs, particularly those with high real-time attentional demands. The 
research will lead to better ways of assessing people's potential to perform 
these tasks and will provide better models for training in these tasks. 



Archival Publications ; 

Posner. M. I.. Choate, L. Rafal, R., and Vaughan, J. (1985). Inhibition of 
return; Neural mechanisms and functior Cognitive N europsychology A^, 
2-11-228. 

Posner, M. I. & Marin, 0. S. M. (Eds.) (1985). Attention an d performance 
XI: Mechanisms of attention . Hillsdale, NJ: Erlbaum. 

Posner, M. I. (1985). Chronometric measures of "g": Commentary on A. R. 
Jenson's: The nature of black-white differences on various psychometric 
tests: Spearman' s hypothesis. The Behavioral and Br ain Sciences, 8, 237-231 

Friedrich, F. J., Walker, J. A. & Posner, M. I. (1985). Effects of parietal 
lesions on visual matching: Implications for reading errors. Cognitive 
Neuropsychology , 213 , 250-264 . 



SKILL AND SKILLED MEMORY 
NR 667-549 

Carnegie-Mellon University/Staszewski 

Scientific Objectives : This research investigates how experts' efficient 
of memory enables them to achieve exceptional performance on difficult an 
demanding tasks. Its goal is to identify and characterize the memory 
structures and processes that experts exploit to access information in an 
extensive knowledge base rapidly and reliably. 
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Approach ; Studies focused on a small number of participants who developed 
expert-level cognitive skills through hundreds of hours of laboratory 
practice. Their skills are analyzed intensively using methodological tools 

analysis, experimental hypothesis testing, and computer 

simulation modelling, 

P|ogress: Two extended training studies investigate the nature and mechanisms 
of expert performance. In both, subjects practiced difficult cognitive tasks 
using strategies identified in prior analyses of established experts In a 
memory training study an otherwise normal participant increased digit-span to 
110 digits with over 800 hours of practice. Approximately 250 hours of mental 
arithmetic practice enabled two other participants to become expert "mental 
calculators who can multiply large numbers with extraordinary speed and 
accuracy without recourse to external memory aids. These studies showed that 
experts capitalized on 1) mnemonic encoding, 2) retrieval structures, and 3) 
practice-related speed-up of encoding and retrieval processes to effectively 
expand their working memory capacity for materials within the domain of their 
expertise. Enhanced working memory, in turn, enabled the experts to overcome 
fundamental Information -processing constraints and achieve exceptional 
performance. 

Potential Applications; This work demonstrates that highly-motivated 
individuals with average abilities can achieve performance levels that most 
people assume require "special" or "exceptional" abilities. Further, detailed 
analyses of experts aimed at identifying both the general cognitive mechanisms 
and domain-specific strategies that support their performance can be used to 
develop effective procedures for training complex cognitive skills. Most 
importantly, the training studies show that the three principles of skilled 
memory — mnemonic encoding, development and use of retrieval structures, and 
speed-up of encoding and retrieval processes through practice ~ represent a 
basic set of design principles for the engineering of expertise in tasks whose 
memory demands severely constrain novices' performance. 



VISUAL THINKING; AN EXPLORATION 
NR 150-480/NR 667-480 
Harvard University/Kosslyn 

Scientific Objectives; This research is exploring new ways of studying the 
nature of the imagery processing system and new ways of testing theories about 
imagery representation and processing. The development of Kosslyn' s theory of 
the processing modules" used in imagery is continued in order to discover 
how processing modules are realized in the two hemispheres of the brain, 

Approach ; Conceptual tools from computer science and methodologies from 
cognitive psychology and neuropsychology are used. In earlier work a 
computer simulation model that could mimic human performance in imagery 
tasks was constructed. This work is testing that theory by using it in a novel 
way; namely, to characterize patterns of behavioral deficits following brain 
damage. * 

Progress; Two patients who had the corpus callosum surgically severed, two 
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aphasics, and a patient suffering from temporal-lobe amnesia have 
been tested in-depth. The theory successfully characterized the data. Two 
especially interesting results were obtained. First, the left cerebral 
hemisphere was superior to the right in some (but not all) imagery ^asks; 
this finding is counterintuitive to many (the right hemisphere has often been 
claimed to be the seat of mental imagery) but was predicted by the theory. 
Second, the processing modules are not independent; damage can disrupt the 
ability to use two modules at once while leaving intact the ability to use 
them individually or in combination with other modules. In addition studies 
were made of processing in normal observers and a new computer-simulation 
model of imagery was developed. The new model was neurologically plausible, 
and was used to predict effects of brain damage, as well as data, from 
normal observers. 

Potenti al Applications ; First, results can guide construction of new 
psychometric instruments, allowing selection of personnel best suited for 
performing tasks requiring different sorts of imagery. Second, it is now 
feasible to train the specific imagery abilities, having characterized them 
well enough to design specific exercises. Third, this characterization of the 
human imagery system suggests ways of building computer-aided imagery systems, 
which essentially "externalize" imagary. . Uowing one to use a computer to 
circumvent weaknesses in the human system (e.g., its low capacity and 
resolution) . 

Reports : 

Kosslyn. S.M.. Cave, C. B., and Provost. D. (1985) Seguential processes in 
image generation ; An objective measure (Tech Rep //6) . Harvard U. . Department 
of Psychology, September 1985. 

Kossyln S.M. (1985). Visual hemisph e ric specialization ; A computational theory 
(Tech Rep.# 87). Harvard U.. Department of Psychology, September 1985. 



THEORETICAL AND EXPERIMENTAL RESEARCH INTO BIOLOGICAL MECHANISMS UNDERLYING 
LEARNING AND MEMORY 
NR 670-462 

Brown University/Cooper 

Scientific Objectives ; The major goal of this project is to elucidate the 
biological mechanisms that underlie learning and memory. Experimental data on 
a cellular level are to provide a basis for functional organizations of large 
numbers of neurons that receive sensory and/or interneuronal information. 
Jms comprehensive program of research includes investigations of biochemical 
principles governing synapsic formation and modification, the dependence of 
learning on synaptic modification and organizational principles for neuron 
network models that can reproduce higher-level cognitive acts. 

Approach - Analytical and experimental approaches are being employed in 
testing t he implications of proposed theoretical mechanisms, and in examining 
the theoretical consequences of experimental results. Principles that appear 
to be operating on the cellular level are used to construct models of higher- 
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level functions. 



Progress ; Ideas about synaptic modification, applied to visual cortex 
gave results that agree with classical experimental data obt'iLd the 
corLf /'^^^"^^^"^ theory to a more realistic anatomy of vfsual 

cortex illuminated synaptic modification dependence on local infoLation 
ILT] "I on certain global properties, permitting further compa^sons with 
experiment Several experimental investigations of global controls were 
completed, including a study of the importance of activity "the 
norepinephrine system for learning during the critical period. Neural models 
muUi^^'b L '° P«^^h°^°8i"l Ph—a such as conm^on associations 

f 1 perception, categorization, feature analysis, and abstraction A 
model of language acquisition that can account for important aspects of 
grammar and language was developed. aspects or 

ne^^us'svtKm'l!!'""' /""'^^^^ °^ ^-^^ P-«sses by which the central 
nervous system learns and organizes itself will have important implications 
for computer science as well as for the understanding of cog^tive processes 
including various aspects of human learning and behavior. processes. 

Archival Publications : 

Anderson, J. A. (1985). WhatHebbs' synapses build. In W Lew «? 
Lehmkuhle. & J. A. Anderson (Eds.). Sy^ptic modif Ltl"n!-nt;ro; L.rr ^.^r. 
and nervous system organization. Hillsdale. NJ: Erlbaum. selectivity . 

llnU I" K. M,, & Ebner, F. F. (1985). An investigation or 

cholinergic circuitry in cat striate cortex using acetylcholinesterase 
histochemistry. Journal of Comparative Neurolo f ry . 234, 411-430. 

Cooper, L. N., Munro, P. W. , & Schofield, C. L. (1985). Neuron selectivity: 

^ r Synaptic modification, n euron selectivity, and n ervous system 
organization. HiHsHpIp, mt. r.iK^,,^ ^^^^^^ 

Hohmann, C. F. Bear, M. F. & Ebner, F. F. (1985). Glutamic acid 
decarboxylase activity decreases in mouse neocortex after lesions of the basal 
forebrain. Brain Research. 33, 165-168. 



NEURAL BASIS OF VISUAL PROCESSING ABILITY 
NR 670-465 

Harvard University /Kosslyn 



Scientific Objectivesr This research aims to revise an existing computational 

llZl^f "^ r"'"' '""^"^ '° incorporate neurophysioTog cllly 

plausible parallel processing networks and to integrate the theory with a 
broader theory of "high level" visual information processing ExLrimental 
studies will be conducted to isolate the brain locations thf^ carJy out ^Se 
processing described in the theory. ^ 
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Ap proach : The theory will be expressed as computer simulation models of the 
processing of high-level visual information. Experiments will be conducted 
with varying populations of brain-damaged Individuals in order to determine 
whether the major information processing operations described in the theory 
can be localized to particular brain locations. 

Progress ; This work began late in FY-85, and no Status Reports have yet been 
issued. 

Potential Applications ; Spatial abilities are central for performance in many 
Navy tasks. This research can provide better understanding ot the foundations 
of those abilities and a sounder basis for personnel selection. 



****** 
THE FOLLOWING '..'ORK UNITS WERE COMPLETED DURING THE PAST YEAR; 



INDIVIDUAL DIFFERENCES IN STRATEGIES FOR PROCESSING SPATIAL INFORMATION 
NR 150-469 

University of Pittsburgh/Cooper 



Scientific Oblectlv es; The objective was to identify and to characterize 
alternative information-processing strategies that underlie the development of 
skilled performance on complex, spatial p.:oblem-solving tasks. A key 
hypothesis being investigated was that a number of strategy-related factors 
contribute to spatial skill: current repertoire of strategies, efficiency 
in using any strategy to which attention is directed, flexibility in 
selecting a strategy appropriate to the current problem-solving environment. 

Approach ; This project employed a combination of objective (time and accuracy 
measurements) and subjective (verbal reports) dependent measures. 
Overall accuracy in problem solving was measured as a function of complexity 
and structure of the spatial problems. In addition, the time spent inspectii.g 
different parts of the problem was measured, as was the number of times 
problem solvers alternate between different problem parts. Both retrospective 
and on-line verbal protocols were recorded from the participants in an effort 
to determine how they believed that they solved each problem. The convergence 
of all of these experimental measures was used as a basis for inferring the 
underlying strategies for performance. 

Results and Conclusions ; Two primary strategies for solving the spatial 
problems were isolated. One strategy, termed the "constructive method. 
Involves the visualization of a three-dimensional model of an object as a tool 
for reasoning about how two-dimensional parts of the sLructure should be 
combined. The second strategy, the "analytic" method, involves the comparison 
of corresponding two-dimensional parts of objects ~ rather than the 
visualization of the object as a whole ~ to determine local correspondences. 
Neither strategy seems to be associated in a simple manner with level of 
problem solving skill; however, the ability to switch from one strategy to the 
other, depending on the type of problem presented, as well as a rich 
repertoire of strategies from which to select, appear to be essential factors 
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In achieving high levels of skill. Evidence was obtained that all participants 
report some three-dimensional, view-dependent information after exposure to 
problems presenting 2-D object depictions. 

Potential Applications; The chief application of the findings is th- 
development of techniques for enhancing the development of spatial skill 
in a training situation. 

Reports ; 



Cooper. LA. (1985). Strategies In processing spatial informatio n; 
Relationships to spatial a_Etltude (Final Report) University of Pittlbttrgh. 
Department of Psychology, October, 1985. s>Durgn, 

Archival Publication ; 

Cooper, L.A., & Mumaw, R.J. (1985). Human spatial aptitude. In R 

Itl^T r^^'/'^^r Individual diff erences in cog n itive processes . 
New York; Academic Press. 



LEARNING FROM CONTEXT AND VERBAL ABILITIES 
NR 154-505 

Yale University/Sternberg 

Scientific Objectives; The objective of the research under this contract 
to understand the information-processing bases of verbal intelligence. 
What differences in mental processing distinguish highly verbally intellieent 
people from the less verbally gifted? 

Approach: One set of experiments involved "task decomposition", i.e., 
interring the mental processing people do in their handling of verbal' 
materials in order to make sense of what they read. A second set of 
experiments involves development of measures that will pinpoint the sources of 
individual differences in people's verbal abilities. In other words, given an 
understanding of the mental representations and processes people use in 
understanding text and particularly new concepts embedded in these texts, how 
can one best measure differences in people's ability to deal with these new 
concepts? A third set of experiments involved training of verbal intellectual 
skills, and particularly, the skills involved in figuring out the meanings of 
unknown words. ° 

Results and Conclusions; A fairly well-specified and useful theory of verbal 
comprehension was proposed and tested. Sets of experiments were completed 
that show (1) The theory of verbal comprehension provided a verv good account 
ot what it was that made some new words harder (or easier) to learn than 
others (task variance); (2) A way to measure individual differences in verbal 
comprehension in a way highly predictive of success in understanding verbal 

of v! K ^ wi!" "'^"'^^y'^ ''"^^^ °^ Intelligence, and (3) The tr. ning 

of verbal skills according to the parameters of the theory and increases in 
scores on tests of learning the meanings of words from conter.t that was of 
the order of 30/, over original test scores. 
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Potential Applications ; This research has developed excellent measures of 
verbal comprehension abilities that can be used In assessment batteries for 
measuring verbal Ini-elllgence. These tests, unlike vocabulary tests typically 
used, do not suffer contamination from extensive differences in prior 
knowledge, which may have little to do with ability, but a lot to do with past 
opportunities. Moreover, it has developed a faining program that can 
significantly and substantially Increase people's skills in figuring out 

in any field requires gaining an understanding of the language of the field, 
ihe training program emphasizes learnlng-to-learn ^^^^^^ ' "^^^".f JJ^f 
factual content, so that individuals can go on to apply the skills they 
learn in everyday environments. 

Archival Publications ; 

nnr^ir^a C T StemberR. R. J., & Ross, B. (1985). Multicausal Inference; 
E;:?;:!lon'of';virerce in^ausally complex situations. Journal of Experimental 
Psychology ; General , 114 , 239-263. 

Sternberg, R. J. (1985). Beyond IQ ; A Trlarchic theory of human intelligence . 
New York: Cambridge University Press. 

Sternberg. R. T. (1985). Implicit theories °f ^^^^f^"" ' ""f ^^^'^ ' ^""^ 
wisdom. Journal of Personality and Social Psychology. 49, 607-627. 

Sternberg, R. J. (1984). A theory of knowledge acquisition in the development 
of verbal concepts. Developmental Review , 4_, 113-138. 
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L. INSTRUCTIONAL THEORY AND METHODS 



Research in this cluster is devoted to the development of ir.structlonal 
theories that support both the design of advanced computer-based crainir.g 
systems and improvement of more traditional training methods. In 1985, an 
Accelerated Research Initiative, Cognitive and Technological Approaches to 
Navy Training, began to provide the resources to substantially expand this 
effort, with emphasis on relationships to naval needs in the area of remedial 
basic skills instruction. 

Individual human tutoring has been shown tc have large, positive effects 
on educational and training outcomes. Tnerefore, the possibility that 
artificial intelligence technology might be used to simulate the effectiveness 
of individual human tutors provides an attractive research goal. The problem 
of artificially intelligent computer-assisted instruction integrates many of 
the basic scientific concerns of the Personnel and Training research program: 
measurement and diagnosis or cognitive skill and knowledge, adaptation to 
individual cognitive abilities, and instructional strategies and tactics. 
Furthermore, intelligent tutoring systems are uniquely appropriate as 
laboratories for the analysis of complex human learning. The precise tuning 
and control of explanations and feedback .n intelligent tutoring systems, 
along with detailed records of trainee performance, provides a novel 
opportunity for investigating questions of instructional theory and motivation 
in realistically complex learning situations. Many of the findings will be 
applicable to traditional classroom instruv^tion, as weU as to automated 
instruction, but the questions could never be effectively investigated in 
traditional classrooms. 

Through the DoD-University Research ^tnzjientatlon Program (DURIP) and 
an associated contract, this clustf.r includes a conmunlty-bulldlng effort to 
provide selected univer "ty-based research efforts with . common set of 
computational resourc- resec^rch on intelligent computer-based, 

instruction. This act Is intended to avoid duplication of effort by 

individual researcher. to more effectively focus research resources on 
questions of learning a^c instruction by minimizing technical ccraputing 
problems. 

Some of the individual contracts in this cluster address questions, also 
of concern to the computer science community, that are broadly sigrif leant for 
Intelligent computer-based instruction, for example: how can knowledge be 
represented and structured in the computer in order to support the flexible 
generation of explanations? How can artificial intelligence approaches to 
learning be used to infer the student's state of knowledge and current rules 
of operation? 

Other contracts focus upon learning and instruction in a wide range of 
tasks and skills which may present very distinct training problems. In some 
skills, such as air-intercept control, the pacing of external events demands a 
high degree of automaticity in cognitive functioning. Problem-solving skills 
such as those required for trouble shooting electronic devices or programming 
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computers are quite different in quality. Attention is also being given 
to research issues relevant to baric skills in reading and mathematics, for 
which remedial training of Navy personnel is sometimes necessary. 

Recent research on the cognitive requirements for such skills is 
discussed in Cluster M (Knowledge and Skill). That research and related work 
provide the basis for intelligent automation of tutoring, of generation of 
exercises and examples, and of other instructional functions. Another focus 
of contracts in this cluster has been cn different forms of instruction to 
teach procedural skills, via text graphics, and computer displays. The 
procedural skills involved in equipment operation and maintenance are of 
particular concern in military training and have not been emphasized in 
other educational research on text design and readability. 

To coordinate work in instructional methods and technology, we have been 
holding annual meetings, in which all the contractors informally brief each 
other on their latest work and plans. These meetings typically include 
presentations by scientists outside the program who are working on re\-*-ed 
problems, and presentations by military scientists investigating related out 
more applied problems. Selected represencatives of other Navy, DoD, and 
federal agencies are invited to attend the meetings and participate in the 
technical discussions. Coordination efforts in this domain have proven 
successful within the Navy and elsewhere. More applied research and 
development in intelligent computer-assisted instruction that builds upon ONR 
research is now being pursued by MPEDC and NTSC as well as the AFHRL and NSF 
Science Education. In December, 1985 a special meeting of researchers doing 
work related to mathematics instruction (Cluster L and Cluster M) was held 
in San Diego to present information about a CNET-sponsored revision of the 
Nivy's remedial mathematics curriculum by NPRDC. Meetings focused on 
text design and procedural instructions were held in June, 1984 at the 
University of Arizona and at Carnegie-Mellon in June, 1985. In March 1986, 
a meeting of researcherr. in intelligent computer-assisted instruction was 
held at the Xerox Palo Alto Research Center, attracting many representatives 
from industry as well as from the military R&D community. 
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A COMPUTER-BASED CONSULTANT FOR INTRODUCTORY COMPUTER PROGRAMMING 
NR 154-492 

Yale University/Soloway 



Scientific Objectives; The intent is to develop ?. set of principles that can 
guide the construction of a key component of an Intelligent Tutoring System 
(ITS); namely, the component that can diagnose student responses and identify 
potential misconceptions. The Issues that surround this key component are 
explored in the domain of computer programming. Thus, their immediate goal 
is to design, build, and evaluate a system that can diagnose a "buggv" computer 
program generated by a novice programmer, and provide an analysis of that 
program to the student. 

Approach; A diagnostic system, PROUST, has been developed and classroom- 
tested. PROUST finds the non-syntactic bugs in students' programs and suggests 
misconceptions that may be causing those bugs. PROUST embodies a theory of 
how expert programmers find bugs and how novice programmers write programs. 
The key to finding bugs in a student's program is PROUST 's abilitv to infer 
what goals the student was attempting to realize in the program. 

Progress; PROUST was designad to analyze a class of moderately complex 
looping programs (between 40-60 lines of code) typically used in introductory 
programming courses. In classroom tests using actual student programs, PROUST 
accurately found approximately 75% of the bugs. This level of performance 
approached that of a human teaching assistant. Moreover, students who were 
given access to PROUST 's analysis of their programs performed significantly 
better on examinations when compared with students who did not have access to 
PROUST'S analyses. 

Potentia l Applications ; Computer programs are complex artifacts. The 
automation of the ability to analyze them and find non-trivial bugs is a 
significant accomplishment. The general approach taken by PROUST is rot 
limited to the domain of computer programs. A PROUST-] ike system can be 
built to work in a wide variety of domains, e.g., debugging students' 
automotive repair strategies. 

Archival Publication ; 

Johnson, W. L. & Soloway, E. (1985). PROUST: Knowledge-based program 
understanding. IEEE Transactions on Software Engineering . March. 
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COMPUTER MODELS OF OPPONENTS FOR TRAINING IN TACTICAL DECISION-MAKING 
NP. 154-486/154-502 
Perceptronlcs, Inc./Chu 



Scientific Objectives ; The goal is to develop, Implement, test, and refine a 
training system based on a cognitive model cf expertise In time-stressed 
tactical planning. 

Approach ; Cognitive models are being adapted and extended for application to 
time-stressed tasks. Such tasks present unique demands for dynamic replannlng 
and problem solving under multiple constraints. A Modified Petri Net 
representational framework is being used to formulate integrated solutions to 
several related problems in the representation and tutoring of dynamic skills. 

Progress : An experimental computer-based testbed has been Implemented to 
provide a simplified interactive simulation of the tactical surface warfare 
environment. The testbed system has been transferred to a LISP machine and 
has been upgraded to a nearly real-time simulation. An expert rule-based 
model has been developed for controlling ship maneuvers, radar/sensor 
selection, and use of chaff and missiles. The modified Petri Net (MPN) 
representational framework has proved useful in visualizing concurrent 
processes, representing multi-level action traces, and dlscoverlnc diagnostic 
relations between classes of errors and situation-based rules. 

Potential Applications ; The modeling framework and testbed architecture can 
be applied generally to the development of training systems for time-stressed 
planning and decision-making tasks in both the military and civilian sectors. 
They can also be used in systems analysis. The use of such systems reduces 
the demand for human agents to play supporting roles in multi-agent 
simulations . 

Funding; NTEC 



DEVELOPMENT OF COMPUTER-BASED INSTRUCTIONAL SYSTEMS FOR TRAINING ESSENTIAL 
COMPONENTS OF READING 
NR 154-519 

BBN Laboratories/Frederlckson 



Scientific Objectives ; The objective is to develop and evaluate instructional 
methods for training critical components of reading skill. Computer-based 
instruction Incorporates continuous dynamic monitoring and feedback to develop 
and automate efficient Jecoding and comprehension pmceJures. A final 
objective is to extend interactive reading theory to include the high-level 
skills needed for analyzing cohesive features of text. 

Approach ; The instructional approach follows an analysis of reading in terms 
of skill components that are functionally interrelated. Components selected 
for training are those that have a broad Impact on reading performance and 
that present particular problems for low-ability adult readers. Training is 
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component-specific, including immediate feedback on efficiency and accuracy of 
performance Where there are hierarchical interactions among components. ' 
training of lower-order components precedes training of higher-order 
components, and integration of skills is explicitly built into the Jater 
learning exercises. 

Zl£Sr£S3: Based on related research on controlled vs. automatic components cf 
cognitive skill, computer-based systems have been developed for training 
reading-skill components in low-ability adult readers. These training 
systems incorporate advanced graphics, synthesized speech, and individualized 
instructional monitoring. Graphic feedback on performance and progress is a 
central design feature. Experimental evaluations of the training systems have 
been used to improve their design, as well as to gain more detailed knowledge 
of the structure and acquisition of reading skills. 

Potential Applications; The instructional methods developed will be useful in 
remedial training programs, and can also be applied in any area of training 
in which automaticity of skill components is essential to effective high-level 
performance. ° 

F unding ; NPRDC, Manpower R&D Programs. 



COGNITIVE PRINCIPLES FOR INSTRUCTIONAL DESIGN 
NR 154-522 

University of California, Berkeley/Reif and Heller 



Scientific Oblectives; The aim is to formulate and test theoretical 
principles for designing instruction. These principles are based on analyses 
of the underlying thought processes leading to good performance on complex 
intellectual tasks. *^ 

Approach ; Explicit instructional models are formulated and then translated 
into detailed computer-based teaching programs. Individual subjects are 
observed before, during, and after interacting with these programs, while some 
of their verbalized thought processes are tape-recorded. These observations 
are used to assess the effects of particular features of the instructional 
models. Analysis of these data leads to refinements of the instructional 
programs and of the underlying theoratical models upon which they are based. 

f^°8^^f • Preliminary teaching models have been developed in the following 
three domains; (1) Teaching the concept "acceleration" (to study the 
Importance of procedural knowledge for interpreting scientific concepts); (2) 
teaching a procedure for describing the forces acting on an object (to study 
the human control processes needed to perform a complex procedure); and (3) 
teaching a computer operating system (to study the role of knowledge 
organization in learning). Assessment instruments have been developed for 
probing a participant's abiHty to interpret the concept "acceleration." 
Prototype computer programs have been developed, for teaching "acceleration" 
and force-description procedure. 
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Potential Applications ; Because the design is systematic and principled, 
based on analyses of underlying humc?n Information processing, the Instruction 
promises to be effective and applicable to classroom teaching as well as to 
the development of Instructional computer software. 

Archival Publications ; 

Relf, F. (1985). Acquiring an effective understanding of scientific 
concepts. In L. H. West & A. L. Pines (Eds), Cognitive structure and 
change (pp. 133-151). New York; Academic Press. 



IMPASSES; KEYS TO A UNIVERSAL THEORY OF THE ACQUISITION AND MENTAL 
REPRESENTATION OF COGNITIVE SKILLS 
NR 154-535 

Xerox Palo Alto Research Center/Brown 
Camegle-Mellon Unlverslty/Van Lehn 

Scientific Objectives; this goal Is to extend the theory of the learning 
expressed In the computer program Sierra, by testing two new hypotheses; 
that learning occurs when students reach Impasses In their problem solving 
procedures, and that procedural skills are best considered as grammars to the 
Interpretation of special notations. The validity of these hypotheses will 
be tested across several content domains In mathematics and science. 

Approach : The computer program Sierra will be modified to Incorporate the two 
hypotheses above. The modified program will be used to generate predictions 
of student behavior under various versions of the theory. These predictions 
will be tested against an existing database on students' learning of 
arithmetic procedures. A series of Instructional experiments will be 
conducted, systematically varying Instructional materials to test specific 
hypotheses underlying the theory of learning. 

Potential Applications ; Much of Naval training Is concerned with teaching 
procedural skills. Previous work by these Investigators has uncovered new 
conditions on the effective design of Instructional materials, especially 
the content and sequencing of lessons. 

Progress : This Is a new contract, and no Status Reports have yet been 
Issued. 



EMC 



137 

l4o 



NETWORK FOR RESEARCH ON INTELLIGENT COMPUTER-BASED TRAINING 
NR 667-524 

University of Pittsburgb/Lesgold 

LISP MACHINES FOR IhTELLIGENT COMPUTER-BASED INSTRUCTION 
NR DIP-252 

University of Pittsburgh/Lesgold 



Scientific Objectives : The goal is to accelerate the progress of 
Instructional theory for intelligent tutoring by establishing an interacting 
research community focusing cognitive research efforts on possibilities for 
intelligent computer-based training. These projects provide training, 
consulting, software tools, examples, and appropriate computer systems, 
freeing participating laboratories to focus on development and testing of 
instruction and training principles relevant to computer-based training. 

Approach ; Facilities are provided for sharing of software, problems, 
approaches, and research results among the projects involved in this work. 
Consulting help and training workshops are available for cognitive scientists 
who are, for the first time, going to be managing their own powerful computer 
systems and having to deal both with an extremely rich intelligent graphics 
programming environment and with issues in network communications, 
maintenance, and systems-programming aspects of the Interlisp-D environment. 
Tools, such as simulation languages and core utilities for intelligent 
tutoring systems, are being developed, together with portable training 
packages for those tools. 

Progress ; A set of supplementary manuals for the Interlisp-D language has 
been developed and circulated to the other laboratories. The "Friendly 
Dandelion Primer" developed here has become part of Xerox's documentation 
distributed with its machines. Workshops have been held to introduce other 
laboratories and DoD personnel to the Interlisp-D environment. Newsletters 
and a phone consulting service have been implemented. Hardware has been 
installed and upgraded at eight other sites. Examples and tools for window/ 
menu oriented programming have been developed and distributed. 
Videotape-plus-computer-lab-environment training packages have been distributed 
to other sites. Additional workshops have been held and are planned. Portable 
packages are in development. Work on specific tutors is vjell underway under 
various funding auspices. The newsletter has been sent frequently to an 
expanding list of recipients. Visiting scholars from US and abroad have been 
trained and have contributed to the project perspective on ITS. Work on an 
object-oriented approach to ITS curriculum promises increased nodularity and 
coding efficiency. 

Potential Applications ; Increased levels of research on bow to construct 
effective intelligent training systems (coaches, tutors, and exploratory 
environments) will potentiate better future training approaches. This project 
aims to accelerate and focus the activities of more specific ventures. 

Funding : ARI , AFHRL, NSF, NIE 
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Archival Publication: 



Bonar, J.G. (1984). Cognition based intelligent tutoring systems. Tn G. 
Salvendy (Ed.), Huiran-Coinputer Interactions . Amsterdam: Elsevier Science 
Publishers B.V. 



DESIGN OF INSTRUCTION MATERIAI.S FOR OPTIMIZING SKILL LEARNING 
NR 667-529 

Carnegie Mellon University /Reder-Charney 



Scienti fic Oblectives : The primary objective of the first two years was to 
develop a formally specified model of how particular kinds of elaborations of 
text content affect participants' behavior ou specific tasks. The first goal was 
to establish that elaborations ir an instruction text facilitate skill 
performance, in contrast to the detrimental effect elaborations have 
previously been found to have on fact learning. The second goal was to 
explore what types of information benefit from elaboration and what forms of 
elaboration are most effective. 

Approach : Various versions of a computer manual are compared experimentally 
to determine how well they help participants acquire and perform a cognitive 
skill: learning to use a personal computer. Some of the studies also 
manipulate how effectively participants can generate their own task-specific 
elaborations when given advance information about the tasks they will have to 
perform on the computer. On-line data-gathering is used to measure bow well 
participants perform tasks on the computer (without having the manual to refer 
to), and how efficiently they work (e.g., how much time they spend working on 
a task and how many steps they take). 

Progress : Four studies on elaborations have been completed. The findings of 
the first two studies indicate that regardless of whether elaborations are 
provided in the manual or generated by the reader, they do facilitate skill 
performance. However, participants who knew about the tasks in advance performed 
better if they read the shorter, unelaborated version of the manual. Results 
from the third study suggest that while both expert and novice computer users 
benefit greatly from elaborations on the exact syntactic form required to 
carry out a procedure, neither group benefits when conceptual points are 
elaborated. 

Potential Applications : By gathering experimental evidence about what kinds 
of elaborations facilitate skill learning for various kinds of learners, this 
research will provide useful design principles to wriLcns cf instructional 
materials. 

Reports : 

Reder, L., Charney, D.. b Morgan, K. (1985) The Role of Elaborations in 
Learning a S!:ill from ai Instructional Text (Tech. Rep. 1). Pittsburgh, PA: 
Carnegie-Mellon University. 
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Reder, I.. (1985). Strategy Selection In Question-Answering (Tech Report 
ONR-85-2). Pittsburgh, PA: Carnegie-Mellon University. 

Reder, L. (1985). Beyond Associations ; Strategic Components in Memory 
R etrieval (Tech. Report ONR-85-3). Pittsburgh. PA: Carnegie-Mellon 
University. 



ACQUISITION AND INSTRUCTION OF PROGRAMMING SKILLS 
NR 667-530 

Ca jnegle-Mellon University/ Anderson 



S cientific Objectives : The objective is to understand how LISP programming 
skills are acquired and how they can be taught effectively. The study of thiF 
skill is being used to test Anderson's learning theory. 

Approach ; Computer simulation models based on Anderson's theory are developed 
to account for the learning and problem-solving protocols. Based on the 
structure of these models, computer-based tutorial systems are developed for 
teaching the skills. These tutorial systems are then varied to explore issues 
in the learning theory and the theory of instruction. Use of tutorial systems 
as an experimental tool has enabled novel tests of issues of skill acquisition 
which extend over tens of hours. 

Progress ; In addition to simulating student protocols, a series of 
experiments has been performed to test and advance various aspects of the 
theory. A major effort has gone into converting the predictions of the theory 
into the design of an intelligent tutor for teaching LISP. The LISP tutor has 
been developed to the point where it exposes the student to about 15 hours of 
instruction and covers most of the major programming constructs in LISP. In 
preliminary evaluations the tutor has been shown to be as effective as human 
tutors and much more effective than traditional classroom instruction. 

Potential Applications ; The methods developed in this research effort can be 
extended to other programming languages like ADA. More generally, there is 
the potential for extending the methodology to teaching many technical skills. 

Reports ; 

Anderson, J.R., (1985). Skill Acquisition ; Compilation of weak-method 
Problem solutions (Tech Rep. ONR 85-1). Pittsburgh; Carnegie-Mellon 
University. 

A rchival Publications ; 

Anderson, J. R. (1985). Production systems, learning and tutoring. In D. K, 
Klahr, P.W. Langley, & R. Neches (eds.), Self-Modifying Production Systems; 
Models of Learning and Development : Bradford Books/MIT. Cambridge, MA. 
1985. 
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Anderson, J. R. . Boyle, C. F. , & Reiser, B. J. (1985). Intelligent tutoring 
systems. Science , 228, 456-467. 

Anderson, J. R. & Jeffries, R. (1985). Novice LISP errors: Undetected losses 
of information from working memory. Human Computer Interaction , J^, 107-131. 

Anderson, J.R. & Reiser, B.J. (1985). The LISP tutor. Byte, ^0, 159-175. 

Lewis, M.W. & Anderson, J.R. (1985). Discrimination of operator schemata in 
problem solving; Procedural learning from examples. Cognitive Psychology, 
r7, 26-65. 

Reiser, B.J., Anderson J.R., & Farrell, R.G. (1985). Dynamic student modeling 
in an intelligent tutor for LISP programming. Proceedings of IJCAI-85. 8-14. 

Singley, K. & Anderson, J.R. (1985). The transfer of texi-editing skill. 
Journal of Man-Machine Interaction, 2_2, 403-423. 



APPLYING ACTIVATION THEORY FOR MODELING TASK INTERFERENCE IN DUAL-TASK 

SITUATIONS 

NR 667-5A6 

University of California, San Diego/Norman and Navon 

Scientific Objectives ; The goal is to evaluate the ability of activation 
theory to explain phenomena of dual- task interference. Such phenomena have 
previously been attributed to "limited attentional resources." 

Approach ; A new experimental method, the optimum-maximum method, will be usee 
to attempt to eliminate certain anchoring artifacts which can be created by 
traditional methods. This new method will be refined and used to provide 
rigorous tests of the relative merits of activation versus resource-limitatioi 
theory. 

Progress ; This is a new contract, and no Status Reports have yet been 
issued. 

Potential Applications ; Activation theory is a type of Parallel-Distributed 
Processing (PDP) modeling framework. PDP models can be applied to the design 
and engineering of special-purpose computing architectures for perceptual 
Information processing and motor control. For example, such architectures 
could be applied to the design of Innovative robotics systems. 



A COMPUTERIZED COMPREHENSIBLE WRITING AID 
NR 667-547 

University of Michigan/Kieras 

sc ientific Objectives ; The objective Is to continue the work in a previous 
project (NE 667-513) to develop a computer-based system that will help 
technical writers prepare more comprehensible technical documents. 
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A2£roach: A previous demonstration system - using natural-language 
processing techniques from artificial intelligence and research results from 
psycholinguistics to identify problems in technical documents that will 
produce comprehension problems for the reader — is being upgraded and 
extended to include a more sophisticated parser, and more detailed 
comprehensibility rules. 

Progress: The system is being developed on Xerox 1108 LISP machines, and a 
basic development environment has been implemented to make the construct ion of 
more complex gratamar easier to do. A more elaborate svstem will be 
lltZlZTt^ ^^^^^^ ^ prototype version of the system. This system will be 
evaluated to see if its output is useful to actual technical writers. 

Potential applications: This project is directly oriented toward developing 
the prototype for a fieldable system to aid technical writers. 



DESIGNING AND HIPLEMENTTNG AND "INTELLIGENT" MULTIMEDIA TUTORING SYSTEM FOR 
REPAIR TASKS 
NR 702-001 

University of Colorado/Baggett 

Scientific Objective: The intended end product of this research project 
«no^?f. Jff" "'"^'^^"■^'^i^ tutoring system for procedural tasks, and more 
specifically, for assembly and repair of physical objects. By multimedia is 
meant moving vide, (from videodisc), still pictures and computer graphics, and 
spoken and written text, all combined with hands-on practice by the user. The 
main goal is to find the theoretical and practical principles for good 
multimedia instructional design, focusing on and i"ntelligent" interactive 
computer system, while also actually developing such a system. 

ApEToach: The theoretical framework guiding this work concerns how people 
process information. It deals with concept formation and with encoding, 
retaining, and using information from multimedia stimuli. Within the 
- framework, a concept is represented as a graph with two kinds of associative 
links (subconcept and pointer), and five types of nodes, corresponding to 
elements from different modalities, 1. e., mental, abstract, motoric (action), 
visual (pictorial), and linguistic. The method of research is to build an 
experimental implementation using the data structures and procedures acting on 
them which are presented in Technical Report 142-ONR. 

|£ogress: 1. In the first phase of the research, participants were given 
four problems, or tasks. The data analysis found what the "natural" queries 
of subjects are when they are given a task. Information access was then 
designed to maximize the success rate and minimize collisions, using 
characteristic elements in the participants' queries as access keys. The 
technique, based not on semantics but on simple pattern matching, seems to be 
abe to provide fast access to large data bases. 2. Using a prototype 
multimedia workstation on loan from IBM. we have run our first experiment on 
assembly. The interactive presentation combines videodisc, graphics, and 
speech as output. Subjects manipulate what they see using touchscreen only as 
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input. They build an object during the presentation and then again from 
memory. The main advantage we observe thus far to interactive multimedia 
instructions over passive video is time to work when working from memory. 
Overall, subjects work less than 70° as lon^ qfter intpr .ctive instruction?. 
3. We are just beginning research into desigulng cur .vit "intelligent' 
multimedia presentation for repair tasks. 

Potential Applictions ; This work should lead to the formulation of 
theoretical and practical principles for designing interactive multimedia 
instructions for procedural learning. Also, an "intelligent" multimedia 
computer system will be developed, which can be used as a prototype in 
training sifuations. 



Reports ; . 

Baggett, P. and Ehrerfeucht, A. (1985). Conceptuali zing in Assembly Tasks. 
(Technical Report 139). The University of Colorado, Department of Psychology, 
April, 1985. 

Baggptt, P. and Ehrenfeucht, A. (1985). A Multimedia Knowledge Representation ± 
Intelligent Computerized Tutor (Technical Report 142) . The University of Colora 
Department of Psychology, April, 1985. 



FLEXIBILITY IN JOINT PROBLEM SOLVING: 
VIEW ON OVERCOMING BLOCKS 
NR 702-002 

University of San Diego/Levin 



THE EFFECTS OF DIFFERENT POINTS OF 



Scientific Objectives ; This work unit is a continuation of work begun under 
IIR 667-532. The goal continues to be the specification of the role that 
conceptual points of view play in problem solving, especially when a problem 
solver is blocked (i.e., does not know how to take the next step). 

Approach ; Computer-based graphics microworlds have been r-eveloped as an 
Instructional and experimental environment for the study ^f pro»'lem-sPlving 
skills. These micro-worlds allow the experimental manipulations of point of 
view as well as the measurement of changes in point of view during problera 
solving. 

Progress ; This is a new contract, and no Status Reports have yet beep 
issued. 

Potential Applications ; Results will be useful in development of practical 
problem-solving aids and intelligent autoii.atcd tutoring systems to teach 
problem-solving skills. 
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rSiNG*^^^° EXPLAINING AND MANAGING THE PROCESS OF DIAGNOSTIC 

NR 702-003 

Stanford University/Clancey 

Scientific Objectives; The hypothesis under Investigation is that a 
diagnostic procedure can be represented separately from the device-specific 
model It operates upon, and such a procedure can be used to simulate expert 
problem-solving behavior, generate explanations of reasoning for a student, 
and serve as a baseline for modeling student performance. 

A£2ro|ch: A family of tutoring programs is being constructed that will enable 
a student to watch, explain, debug, and learn in apprenticeship the process of 

Jer??t°pJ'^», ^^''r'"^"''!/" expert-system shell, called 

HER/.CLES. that solves problems by the heuristic classification method. 
Modeling and explanation programs relate the student's behavior to the 
idealized model, attempting to sort out discrepancies in domain knowledge from 
discrepancies in problem-solving strategy. Synthetic students (systematic 
variations to the ideal model) and experiments with actual students are used 
to develop modeling capabilities. The family of tutoring programs 
systematically explores the advantages of different sc-narlos for learninK 
(for example, critiquing another student's explanation versus completing a 
partial solution), allowing determination of how these activities might be 
sequenced and combined. 

Progress ; A medical diagnosis program, called NEOMYCIN, has been developed 
that performs well on a set of test problems to be used in teaching. The 
generality of the diagnostic model has been demonstrated by implementation of 
a prototype program for diagnosing defects in iron cast in sand. The first 
tutoring program. GUIDON-WATCH, incorporating strategic explanations and 

r«^rnira ^^^r "ff"i"8' i« operational and being tried with students. 
A second generation modeling program, called ODYSSEUS, is partially 
implemented; it cau recognize alternative but non-optimal paths of reasoning, 
and Interact with the student to diagnose the cause for discrepancies in 
behavior. A new explanation program now under development uses a model of the 
user domain knowledge and Improved implementation of the diagnostic strategy 
to more selectively mention facts used in reasoning. GUIDON-DEBUG, a program 
that allows a student to critique and Improve someone else's solution, is in 
the early stages of implementation. 

!xne;L'nM^r.'''^?T \ P"^"'^ " fran^^work for representing 

experiential knowledge (contrasted with device simulatioa models) in an expert 
consultation system, with advances in ease of system construction and 
?hP ^ Improved explanation over simpler rule-based approaches. 

The heuristic classification model is applicable to a wide variety of 
diagnostic, monitoring, repair, and planning problems. 



erJc 



144 

15^ 



Reports ; 

Clancey, W.J. (1985). Review of Sowa's "Conceptual Structures" (Tech. Rep. 
ONR-TK-11). Stanford University. 

Clancey, W.J. (1985). Heuristic Classification (Tech. Rep. ONR-TR-12) Stanford 
University. 

Clancey, W.J. (1985). Acquiring. Representing, and Evaluating a Competence Model 
of Diagnostic Strategy (Tech. Rep. ONR-TR-13) . Stanford University. 

Clancey, W.J. (1985). GUIDON-WATCH; A Graphic Interface for Viewing a K nowledge- 
based System (Tech. Rep. ONR-TR-14) ., Stanford University. 



EXPERT PLANNING PROCESSES IN WRITING 
NR 702-005 

Camegle-Mellon Unlverslty/Hayes 



Scientific Oblectl" ?s; The goal is to refine and test a general theory of the 
cognitive processes underlying writing, especially processes of goal setting, 
goal management, and planning. The work alms to specify the subprocesses 
of planning (goal setting, goal generation, and organization) and to determine 
how novice and expert writers differ in their ability to adapt, develop, and 
revise their plans during the writing process. 

Approach ; A series of experiments is being carried out, using innovative 
methods of eliciting planning protocols from novice and expert writers. 
Different participants performance will be compared on identical complex planning 
tasks in order to Isolate planning skills from other writing skills. Expert 
and novice plans will be compared at several different theoretical levels, 
ranging from global to very specific. 

Progress; This is a new contract, and no Status Reports have yet been 
Issued. 

u Potential Applications ; To deal with the Increased demand to teach highly 
Technical skills to technically unsophisticated trainees, both the criteria 
for training material and aids for writers need to be based on a sound theory 
of the writing process Itself. The results of this research will clarify the 
processes Involved in planning, organizing, and generating effective 
instructional materials for technical tasks. 



LEARNING AND TEACHING IN THE CONTEXT OF MULTIPLE KNOWLEDGE SOURCES 
NR 702-006 

University of California, Irvlne/Klbler 

Scientific Oblectlves ; The goal is to test a computational theory of the 
knowledge structures and processes associated with naive problem solving, 
expert problem solving, acquisition ot problem-solving skills, and Instruc 
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of such skills. 



Approach; A computational model has been designed for thp knowledge 
structures and cognitive processes underlying skill and skill acquisition In 
the domain of algebra word-problem solving. The computational model rests on 
a number of testable hypotheses about what problem-solving skill Is and how It 
can be learned. These hypotheses are being tested by means of psychological 
experiments and computer simulations. Central to the model are hypotheses 
about problem understanding, particularly the use and Integration of multiple 
knowledge sources. 

Progress: This Is a new contract, and no Status Reports have yet been Issued. 

Potential Applications; Current teaching In mathematics and science Is not 
founded on any rigorous model of learning. The present work proposes such a 
theory. To the extent that this theory Is verified experimentally. It will 
contribute to better design of . .thods for Instruction and assessment In 
technical domains which depend on mathematical problem solving. 



TUTORING EXPERTISE; HUMAN AND COMPUTER 
NR 702-007 

Yale Unlverslty/Soloway 

Scientific Objectives; The goal is to test a computational theory of the 
knowledge structures and processes associated with expert tutorial skill in 
the domain of computer programming. A model for the diagnostic reasoning 
tutorial planning skills of effective human tutors will be used to create 
principles for the design of more effective intelligent tutoring systems. 

Approach ; A theory of plan coordination and plan integration is being tested 
in the domain of introductory Pascal programming. Methodology includes 
computer simulation of human performance, protocol analysis of human-human 
tutorial dialogues, and the analysis of a database of thousands of student 
programming examples. 

Progress ; An intelligent tutoring system for the Pascal programming domain 
has been implemented and is currently being tested in actual classroom 
settings. In a recent test, students who had access to the intelligent 
tutoring system showed measurable gains in terms of "bugs" eliminated from 
their programs and in mid-term grades, as compared with a control group 
matched for number and type of "bugs" initially present. 

Potential Applications; The general plan- integration model underlying this 
research is applicable to many domains other than computer programming. It 
can be applied to any problem-solving or design domain in which complex 
performance produces great individual variability due to the need to integrate 
different plans and goals into a single solution or design. These domains 
range from architecture to circuit design to tactical planning. 
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Report ; 

Soloway, K. (1985). A computer-based consultant for introduc tory computer 
Programililng . New Haven, CT: rale University, Department of Computer Science 

Archival Publications ; 

Johnson. W., Soloway. E. (1985). PROUST: Knowledge-based program 
understanding. IEEE Transactions on Software Engineering. (March). 

Johnson. W., & Soloway. E. (1985). Automatic bug-detection. B^te (April). 



SCHEMAS IN PROBLEM SOLVING: AN INTEGRATED MODEL OF MEMORY. LEARNING. AND 

INSTRUCTION 

NR 702-010 

San Diego State University /Marshall 



Scient ific Objectives : The project has two goals: (1) to examine the 
components of basic schemas used in solving simple arithmetic story-problems 
and (2) to develop instructional procedures that facilitate the development of 
these schemas in novice and/or unsuccessful remedial problem solvers. 

Approach: Mental models of schema creation and use will be developed. These 
in turn will form the basis of computer simulations of learning. Additional 
tests of the models will come from implementation of the theory and related 
materials in a remedial junior-college class. 

Pr ogress: This research has just begun (September 1985). Preliminary studies 
appear to have identified a relevant set of basic schemas. Work is under way 
to determine whether that set is also sufficient to cover all arithmetic 
story-problems . 

Potential Applications: The research will contribute to understanding of 
the knowledge structures used by students to solve story problems. It will 
also provide a theoretically-based system of instruction to develop the 
necessary structures. This should be of interest in the Navy's Remedial 
Mathematics Training of recruits. 



THE IMPACT OF AN INTELLIGENT COMPUTER-BASED TUTOR ON CLASSROOM SOCIAL 
PROCESSES: Afl ETHNOGRAPHIC STUDY 
NR 702-013 

University of Pittsburgh/Schof ield 

Scie ntific Objectives : The go?] of this project is to assess the Impact of 
the introduction of an intelligent computer-based tutor on classroom struct 
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and functioning. Particular attention is being paid to issues such as (a) 
the new demands which this innovation places on students and teachers, and 
(b) how both students and teachers react to these demands. 

Approach; Ar intensive ethnographic study of classes which use the 
intelligent computer-based tutor, and similar classes which do not, is being 
carried out. The two basic data-gathering tools are qualitative observation 
in these classrooms and repeated interviews with both students and teachers 
Since observations were also made before introduction of the computer-based' 
tutor l>oth longitudinal and cross-sectional comparisons are being made. 

Progress; In the few months since work began, we have gained access to all 
classrooms necessary for the study and begun weekly observations. Sampling of 
students and teachers for the interview panels has been completed and the 
student interview instrument has been developed and is now being pretested. 
Developmental work on the interviews with teachers Is also well underway. 

P.otential Applications; Findings should be of considerable use in suggesting 
effective and efficient ways of utilising intelligent computer-based tutors 
in classroom settings. 



ENHANCING INCREMENTAL LEARNING PROCESSES WITH KNOWLEDGE-BASED SYSTEMS 
NR 702-016 

University of Colorado/Fischer 

Scientific Objectives; This research effort aims to understand, how people 
Incrementally learn to be expert in operating complex systems, and how they can 
be assisted in becoming more expert in the course of their operational 
experience. The particular case of computer systems for knowledge-based AI 
programming will be investigated. 

Approach: Increasingly complex subsets of knowledge used in operating a 
complex knowledge-based computer systeji will be defined based upon records of 
user behavior. AI inference techniques will be used to diagnose user 
expertise in those terms. Building upon an existing AI system that can 
suggest improvements in LISP code, instructional interventions designed to 
improve system knowledge will be attempted and tested for their effects on 
subsequent performance. 

Progress; This is a new contract, and no Status Reports have yet been issued. 

Potential Applications: This research should yield insights into the way manv 
complex computer-operated systems in the military could be designed to provide 
automated, embedded on-the-job training to bring operators to full «!kill 
levels. 
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THE FOLLOWING WORK UNITS WERE COMPLETED DURING THE PAST YEAR: 



BASIC PROCESSES AND INDIVIDUAI DIFFERENCES IN UNDERSTANDING AND USING 

INSTRUCTIONS 

NR 154-461 

Bolt, Beranek and Newman, Inc. /Smith & Spoehr 



Scientific Oblectlves ; The goal was to examine Individual differences In the 
comprehension and use of written Instructions for procedural tasks. Critical 
questions included; (1) Is there an optimal order In which to mention the 
various topics In a step? (2) Are different ways of organizing a fixed set of 
steps best for different types of tasks? 

Approach ; Participants executed procedural Instructions about operating a 
control panel. They executed the instructions one step at a time and dependent 
measures Included the time to read the instruction, the time to execute it, 
and the accuracy with which it was executed. The major independent variable 
was the order in which the various topics in a step are mentioned, e.g., either 
the main action is given first or topics are mentioned in the order in 
the main action is given first or topics are mentioned in the order in which 
they will be needed during axecutlon. Other Independent variables included 
the amount of practice subjects have with a particular order of topics and 
individual differences in subjects' working memories. 



Results and Conclusions ; Results showed that, because information from a step 
is comprehended by fitting it into a plan for carrying out the step, the 
optimal order of presentation for material in an individual step is the order 
in which the material is needed when the step is executed. Subjects are able 
to develop facllltatlve reading strategies for non-optimal wordings of steps 
if given sufficient practice. In addition, it appears that some of the 
individual differences in understanding instructions may be due to differences 
in the capacity of working memory. Having learned a "mental model" for a 
device facilitated memory about device components, instruction reading time, 
and ability to reason about the device. 

Potential Applications ; Findings about the optimal order in which to 
mention the topic in a step may lead to general guidelines for the 
construction of instruction manuals, or perhaps more significantly, the 
structure of instructions delivered by automated maintenance aids, such as the 
Army's PEAM system. Findings about individual differences in working 
memory could lead to a diagnostic for determining how well different people 
will do in different kinds of tasks. 
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PRINCIPLES OF INTELLIGENT COMPUTER-BASED INSTRUCTION por g 
NR 15A-A97 

University of Pittsburgh/Greeno & Brovm 



§uScientiflc Objectlvesllu: The goa] of this project was to identify, formulate, 
and evaluate principles of instructional design based on research in cognitive 
science. Attention was focused on two general topics: instruction of basic 
skills, and acquisition of knowledge for representing problems. 

°§u)roach1(u: Results of cognitive analyses of instructional tasks were used in 
developing instructional systems. The research included development of 
learning tasks and implementation of the tasks in computer-based instructional 
systems. The tasks focused on significant components of cognitive skill that 
usually are embedded implicitly in instr .ction. Participants were observed 
working on these learning tasks, and conclusions were reached about 
acquisition of the skill components that have been identified in the cognitive 
analyses.- The number of participants was small — usuallv 5-10 ~ ob^^erved 

"^^^ recorded protocols. Participants in studies of algebra included 
both middle-school students and adults recruited from college-age students 
having difficulty in an elementary course. 

§uResults and Conclusionsllu: The existing designs of intelligent tutors were 
extended to incorporate new aspects of cognitive skills. One example is 
the cognitive structure of a procedure. Rules and strategies for 
problem-solving in algebra were taught in a task of transforming arithmetic 
expressions. The structure of the tutorial system focused on basic cognitive 
skills of applying operators correctly, rather than general strategies. 
Another example is knowledge for representing problems. A graphics system was 
used by students to create explicit representations of the quantities in word 
problems, and the compositional relations among those quantities. Learning to 
use the system, students reached a deeper understanding of relations between 
formal algebraic notation and concrete quantitative structures. Progress was 
also made on processes of reflection on a problem-solving record, and 
relations between concepts and problem-solving operations. 

Potential Applications: At a general level, these results provide examples of 
ways in which results of cognitive research can be translated into 
instructional materials. The analyses of tasks in algebra and statistics 
Identified specific cognitive processes needed for successful learning, and 
Instruction focused on those processes has been designed. Similar analyses in 
other domains should be made easier by the availability of these results 
More specifically,* the instructional systems developed for training basic 
skills and representational knowledge in algebra could be reimplementod into 
instructional materials for basic mathematical training in the military or 
other educational settings. 

Report : 

Greeno, J. G. Brown. J. S.. Foss. C. Shalin.V.. Bee. H. Y.. Lewis. M.W. & 
Vitolo, T.M. (1985). Cognitive Pri nciples of Problem-solving and Instructio n 
(Final Report). Pittsburgh. PA: University of Pittsburgh. Psychology Department. 



ERIC 



OPTIMAL STRUCTURES FOR MULTIMEDIA INSTRUCTION 
NR 154-500 

SRI International /Goguea 

Scientific Objectives : The goal was to produce effective multimedia 
instructional material, particularly computer-based material, by using 
experimentally verified principles for choosing among the many possible 
instructional approaches permitted within interactive visual and verbal media. 

Approach ; The research paradigm was a combination of experimental psychology, 
discourse analysis, and semiotics (the formal theory of sign-systems). 
Psychological procedures were used to examine the effects on comprehension and 
learning, of varying explanation strategies. Linguistic methods were used to 
describe the structure of explanatory discourse. Semiotic theory was used to 
describe the mathematical relations among different representational systems 
in an instructional situation, such as spoken and written language, charts, 
diagrams, devices and simulations. 

Results and Conclusions : The major findings include the following: Choice of 
discourse type can strongly affect comprehension in ways that would not be 
predictable from other existing theories of instruction or discourse. Visual 
icons can greatly aid comprehension even in the absence of optimal coordinatio 
with verbal instructions. Manipulation of fine-grained discourse structure 
has significant and systematic effects on learning. The design of. computer 
graphics should parallel the structure of successful naturally-occurring 
multimedia interaction, using integrated, hierarchically-structured icons 
with a rich set of indices into and out of the task domain, with clearly 
marked boundaries. 

uPotential Applications : These results are applicable to the design of 
multimedia explanations of technical subjects, including videotaped 
demonstrations by a human tutor and computer and simulation-based training 
Other applications include the design of natural- language and interactive 
graphical human- computer interfaces for intelligent tutoring systems and 
on-board fault diagnosis systems. 

Report : 

Goguen, J., Linde, C, & Bikson, T. K. (1985). Optimal structures for 
multimedia instruction (Tech. Rep. No. 4778). Menlo Park, CA: SRI 
International. 



MODELING THE STRATEGIES OF MATHEMATICS STUDENTS 
NR 154-508 

Carnegie-Mellon University/Langley 

Scientific Obiectives : The goal was to find techniques to automate the 
diagnosis of (possibly erroneous) problem-solving strategies used by student 
on the basis of their performance. 



151 



Approach; The diagnostic problem was divided into three parts: To identify 
the problem-solving steps taken, to identify the order in which those steps 
were taken, and to identify the rules which generated those steps in that 
order. The steps involved were founti ly applying all possible steps backwards 
from the answer given by the student. The solution path was found by 
selectively searching through the possible orderings of these steps on the 
basis of psychological theory. The strategy was found by abstraction over 
several solution paths. 

Results & Conclusions; A computer program implementing the techniques 
described above is completed. It has been applies to small sets of data in 
the domain of multi-column subtraction and it successfully diagnosed both the 
correct subtraction strategy and a few types of misconceptions. 

Potential Applications ; General, computer-implemented techniques for 
cognitive diagnosis could be applied to help provide individualized 
instruction. 

Archival Publications; 

Langley, P. (1985). Learning to search; From weak methods to domain-specific 
heuristics. Cognitive Science . 9, 217-260. 



COMPUTATIONAL THEORIES OF SKILL ACQUISITION AND PROBLEM SOLVING 
NR 667-477 

Xerox Corporation/Brown & VanLehn 

Scientific Objectives ; This research tested a theory based on two main 
hypotheses; (1) that generalization of examples suffices to explain human 
skill acquisition, provided that the student utilizes the information inherent 
in the organization of the lesson sequence, and (2) that people execute their 
procedures reflectively, by modifying their execution state in certain ways 
not prescribed by the procedure. 

Approach; The research strategy was to fit models of learning and problem- 
solving based on artificial intelligence technology into the traditional 
criteria for scientific theories. The theory' s hypotheses were formalized, 
along with competing hypotheses. Each version of the program was run on 
simulated lessons. The output, a set of predicted error types, was compared 
to error data from students taking those lessons. A hypothesis was accepted 
if its predictions were better than the predictions of the competing 
hypothesis. 

Results a nd Conclusions ; A variety of knowledge structures based on 
traditional computer-science notions of procedures has been tested, using data 
from students of several basic mathematical skills. Recently, the 
investigation has focused on revised notions of procedural knowledge. As a 
by-product of this research, special computational tools have been developed 
to manage the hypotheses, the predictions, and the arguments connecting them. 
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Potential Applications ; The theory-managing tools can be used by any 
cognitive scientist with the appropriate computer resources. When complete, 
the thcorv may help Improve Instruction by predicting "mind bugs" and other 
misconceptions, by evaluating £»nd fine-tuning curricula, and by suggesting 
effective lesson sequences. 

Archival Publication ; . 

J.S. Brown (1985). Idea-amplifiers ~ new kinds of electronic learning. 
Educational Horizons, 63, No. 3. 



AN AUGMENTED COMPUTERIZED READABILITY EDITING SYSTEM 
NR 667-513 

University of Michigan/Kleras 



aid 



Scientific Obiectives ; The objective was to use existing psycholinpuistic 
data to develop principles for the design of an automated text-analysis 
system. The system should be able to critique and suggest revisions for a 
typical range of technical text. 

Approach ; The research plan combined an extensive analysis of the existing 
literature in psycholinguistics with natural language comprehension methods 
derived from artificial intelligence research. The combination of these ^ two 
approaches yielded a principled design for an automated technical writer's a__ 
that incorporates a deeper understanding of comprehension and natural language 
structure than do conventional writers' aids. 

Results and Conclusions ; A large number of psycholinguistics articles was 
summarized, and an initial set of comprehensibility rules was devised. An 
experiment has been conducted in which the quality of a simulated technical 
manual was improved by using these rules. A demonstration version of the 
comprehensibility system was constructed using components from earlier 
comprehension models. The demonstration system was able to make many useful 
comments about maintaining coherence and consistent terminology. 

Potential Applications ; Development of an actual usable system to improve the 
content of technical material. 



Report g ^; 

Kieras, D. E. (1985). The Potential for Advanced Computerized Aids for 
Comprehensible Writing of Technical Documents (TR-85/ONR-17) . Ann Arbor, MI, 
University of Michigan, College of Engineering. 

Kieras, D. E. (1985). Improving the Comprehensibility of a Simulated 
Te chnical Manual (TR-85/0NR-20) . Ann Arbor, MI; University of Michigan, 
College of Engineering. 
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Kieras, P.E., & Dechert, (1985). Rules for Comprehensible Technical Prose: 
A Survey o f the Psvcholinguisti c Literature (TR-85/0NR-21 ) . Ann Arbor, MI: 
UniverFity of Michig;?n, College of Fngineering. 

Kieras, D. F. (1985^. An augmented computerized readability edi ting system: 

Final report (TR-85/ONR-2Tr Univorf^ity of Michigan, College of 

Engineering, 



FLEXTRTLTTY TN JOINT PROBLEM SOLVING: THE EFFECTS OF DIFFERENT POINTS OF VIFW 
ON OVERCOMING BLOCKS 
NR 667-532 

University of California, San Diego/Levin 



Scientifi c Objectives ; The goal was to specify the role that conceptual 
points of view play in problem solving, especially when a problem solver is 
blocked (i.e., does not know how to take the next step). 

Approach: Computer-based graphics microworlds have been developed as an 
instructional and experimental environment for the study of problem-solving 
skills. These microworlds allow the experimental manipulation of point of 
view, as well as the measurement of changes in point of view during problem 
solving. Experiments have been run using these graphical microworlds and a 
to look at the role of point of view in individual and group problem solving. 

Results and Conclusions ; Several programs have been written to implement 
microworlds for learning and problem solving. These programs have been used 
in several experiments on the role of point of view in problem solving. 

Potential Applications ; The results of this research will be useful in the 
development of practical problem-solving aids and intelligent automated 
tutoring systems to teach problem solving skills. 

Archival Publications : 

Levin, J. A., & Miyake, N. (1985). Care and repair of a microcomputer. 
Electronic Learni ng, Sept. 



This work is being continued under work unit NR 702-002. 
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M. KNOWLEDGE AND SKILL 



This cluster Is aimed at characterizing skilled perforniance in 
rcf^l-world tasks which deinand complex information processing. Map 
interpretation, troubleshooting, and computer programming are three of the 
many possible examplos of such tasks in the Navy. Expertise in these kirds of 
tasks Is often rare and acquired vlth considerable difficulty after long 
post-instriirtion experience. Furthermore, the cognitive processes involved in 
these tasks are commonly unavailable to the consciousness of an expert, fo 
that explicit accounts of these processes arc not available for instruction of 
novices or for other purposes. Demands for high levels of expertise are 
increasing in the Navy along with the pressures on training, at the same time 
that the manpower pool is decreasing. This research provides explicit formal 
theories of crucial cognitive components of required skills. These theories 
constitute the foundations for selection of individual r with export potential, 
training methods which can shorten the time now required for skill 
development, and personnel-evaluation techniques for assessing an individual' s 
level of expertise. v 

The general approach of this research is to build models of skilled 
performance which directly represent the relevant cognitive processes and 
structures. In addition, these models will have parameters which reflect the 
characteristics of the individual problem solver, his or her training history, 
and the problem-solving environment. 

During its five-year history, this cluster has witnessed the development 
of several significant results avid trends which go beyond those derived from 
individual efforts. One general development is a body of results on the 
differences *>etween expert and novice planners and problem solvers. 
Investigations of experts and novices in software design, medicine, physics, 
and other fields show that both classes of individuals solve problems by a 
combination of planning by abstraction and means-ends analysis. What gives 
the expert an advantage is a knowledge of problem types and the appropriate 
memory structures for planning by abstraction. 

Results of a more theoretical nature are concerned with the acquisition of 
problem-solving and trouble-shooting skills. Theories and models have been 
developed to define problem domains and their :^nterrelations in terms o^ the 
knowledge required to solve problems in each. Computer simulations have been 
developed which model the acquisition of problem-solving skills via rote 
learning, insight, exercise, and example. There is every reason to believe 
that these theoretical developments can be extended to provide a theory of 
learning prerequisites, to apply it to the empirical work on expertise 
discussed above, and to provide methods for curriculum design. 

Coordination among contractors in this area is facilitated by annual 
contractors' meetings. Meetings have been held at Berkeley in June, 1984, 
and the University of Washington in May, 1985. The next in this series is 
planned for the University of Illinois in July, 1986. 
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^iSJSi^^f '^^^^^'^ SELECTION AND SKILL TRAINING FOR HIGH WORKLOAD 

JiNViRONnENTS 

NR 154-527 

University of Pittsburgh/Schneider 

Scientific Objectives; The work seeks to ^mprove our understanding of skill 
acquisition and to develop more efficient computerized training procedures. 
The research applies automatic/controlled processing concepts to building 
complex information processing capabilities by incorporating modular, 
semi-automatic components into multiple processes. 

Approach ; The approach includes quantitative modeling and empirical testing. 
Previous empirical work has shown that automatic processes develop after 
extensive practice and are fast, parallel, require little attentional effort, 
and are difficult to modify. Here the empirical work is examining these 
findings in the context of microprocessor-based training of perceptual tasks 
(i.e., air-intercept control) and cognitive tasks (i.e., electronic 
troubleshooting and semantic search). The studies examine learning ratr and 
transfer as a function of computer sequencing, training workload, computer 
graphics, spacing, and training trials. The theoretical work is modeling the 
learning process with a novel combination of concepts from multiple-access 
communication theory, distributed memory models, and production systems. The 
modeling provides quantitative predictions and interpretations of the 
qualitative changes that occur during training. 

Progress; Empirical results illustrate that extended practice on 
component tasks is necessary before that component skill can be incorporated 
in a higher level skill. Without such training students can not deal with 
the memory load of performing the complex task. These results illustrate the 
functional relationship between resource load and practice. 

Potential Applications; The procedures are being tested in training air 
intercept control and electronic troubleshooting. The training programs may 
provide more efficient training of high workload skills. 



GENERAL PATTERNS OF INFERENCE 
NR 667-494 

Camegie-Mellop University/Larkin & Carbonell 

Scientific Objectives; The goals are to produce a prototype computer- 
implemented system that would use hunan-like general patterns of inference 
(together with some domain-specific knowledge) to solve a large group of 
problems and to teach problem-solving skills. 

Approach; Object-based programming is used to create a hierarchy of both 
procedural and declarative knowledge. Therefore, general knowledge can be 
encoded just once and then used in a variety of situations. 

Progress: A running system called "FERMI" demonstrates the feasibility of the 
approach. The structure of FERMI suggests that most of the knowledge 
associated with general scientific principles is procedural rather than 
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declarative. This system forms the basis for experiments on how to teach 
humans general reasoning procedures. Current work is first exploring how 
FERMI cm be extended to solve new sets of problems, and how it can use 
altern-ive mathematical strategies in doing so. Second, we are beginning to 
explore how FERMI 's organization of knowledge .n facilitate human learning. 

Potential Applications ; The ideas in FERMI' s design could be used reduce 
redundancy and improve performance in expert systems and in instructional 
systems based on expert systems. 



REM. TIME ASPECTS OF THINKING AND PROBLEM SOLVING 
NR 667-528 

University of Washington/Hunt 

Scientific Ob jectives ; Previous work (under NR 150-^57, Performance in Dual 
Tasks) provided a model of problem solving under real-time pressures. This 
project extends the model to the analysis of problem solving while a person is 
simultaneously operating (simulated) machinery. A search is being made for 
parameters of the model representing individual pe-formance in different 
attention-demanding situations. 

Ap proach ; A computer program simulation of real-time problem solving has been 
construc ted. Other programs are being constructed to mimic the operation of 
simplified machinery, analogous to a "computer game" requiring problem solving 
and rapid decision-making. The simulation program is used to analyze how a 
person switches attention between different aspects of the machinery 
operation" and problem-solving tasks. The model's predictions are compared to 
the data obtained from young adults who operate the "machinery, with special 
interest in consistent individual differences in the ability to allocate 
attention when faced with different types of machinery operation tasks. 

Progress; A simulation program developed during the previous repo.'ting period 
was used to model a wide variety of experiments in which people must decide to 
attend to one aspect of a stimulus while ignoring another, that would normally 
lead to an incompatible response. This addressed a basic issue in psychology, 
the question of response selection in situations where different but 
incompatible responses are possible. An extension of the same simulation was 
used to model a situation in which people did arithmetic problems while 
listening for the occurrence of a sound. On hearing the sound they . 'd to 
respond by pressing a button, and then return to the arithmetic task. The 
simulation recovered the pattern of interference in activity associated with 
information load in the sound and the arithmetic task. The two simulations 
indicated that there should be a general "ability" to control one s attention 
which would not be specific to stimulus modality. A series of experiments was 
done in which people had tc -hlft their attention from one to another P^rt ot 
the vl«,ual field, or from one to another part of the auditory field, or had to 
shift from doing one arithmetic task (addition) to another (multiplication) . 
A common attention shifting ability was found that generalized across these 
tasks . 
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Potential Applications ; In the long-term, a theory of real-tlne problem 
solving could be used to estimate mental workload during the design phase of 
man-machine systems. The experimental procedures we have developed could be 
used to identify people who cope well with multiple signals under high 
time-pressure. 

Reports ; 

??rn«B*,m^^^^y'^'°": P roblem solving Under Time Constr.^n^. 
(TR-ONR-10). University of Washington. Psychology Department . August. 

Hunt, E. (1985). Science Tec hnology, and Intelligence (TR-ONR-Q^. University 
of Washington. Psychology Department. August. 

Hunt. E. (1985). The Design of Ballantines (TR-0NR-8>. University of 
Washington, Psychology Department. August. 



GENERATIVE PROCESSES IN REPRESENTATIONS OF PROBLEMS 
NR 667-534 

University of Pittsburgh/Greeno & Resnick 



Scientific Objectives; The goal of this project is to determino the 
consistency and t.heoretical coherence of participants' representations of physical 
phenomena and the processes involved in modifying these repiesentations. (See 
also NF 067-544.) 

Approach; Interview methods are used to determine the degref^ to which 
individuals hold consistent, "theory-like" beliefs across a -iriety of test 
situations. Subjects are asked to make predictions and give explanations for 
a set of situations that should evoke key principles regarding motion. 
Various forms of feedback designed to evoke consistency among explanations are 
then given and subjects are invited to review explanations. These conditions 
are included to assess the effects of different forms of intervention on 
restructuring knowledge and modifying explanations. Under some conditions, 
subjects take a delayed post-test to examine the longevity of restructuring 
provoked by feedback. Analysis of responses before and after feedback allows 
us to examine the extent to which general theories are guiding representations 
and the effects of intervention on restructuring. 

Progress; Preliminary findings indicate that participants' detection of 
inconsistency in their own explanations depends largely on their perception of 
similarity between problem situations. Learners' analyses seem to be tied 
close y to problem conditions; different conditions evoke separate explanatory 
principles, and learners may not perceive their responses as inconsistent. 
in addition, when participants are shown how two predictions are theoreticallv 
inconsistent, they will often protect and maintain their original explanatW 
by appealing to differentiating situational variables. These'carly findings 
are suggesting a reconceptual ization of the theory-driven versus 
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local-explanation questions with which we began. In particular, they 
point to the need for a formulation that focuses as much on processes of 
explanation formation as on the theoretical power of the particular principles 
that are involved. 

Potent ial Applications ; Findings will contribute to the refinement of 
cognitive theories of learning by clarifying processes by which complete 
structures of knowledge are acquired nd modified. We hope to make signi- 
ficant contributions to instructional practice through identifying 
Interventions that are effective in inducing major modifications in cognitive 
structures as well as sources of resistance to modification. 



SKILLED MEMORY 
NR 667-536 

University of Colorado at Boulder/Ericsson 

Scien tific Objecflves ; To elaborate a theory of the real-world memory skills 
associated with expertise In a variety of domains. The research will 
determine how experts can rapidly store and recover large amounts of 
information, and determine how general or content-specific are their memory 
ski lis. 

Approach: A number of real-world memory skills are being studied, including 
actors' memorv for their parts, waiters' memory for dinner orders, and 
blindfolded chess. For each skill, a series of laboratory tasks are 
constructed that allow us to measure storage and retrieval times for domain- 
specific information. The contents of retrieval cues are systematically 
varied to assess the structure of the memory representation as reflected by 
latency and accuracy data. In addition, retrospective verbal reports on 
intermediate 3teps in the retrieval process are analyzed. 

Pioeress: Several experiments investigating the organization of long-term 
memory were carried out. In one of these, actors with well-learned parts were 
given brief segments of their parts as cues, and were asked to report the word 
that iiranediately followed or preceded the presented segment. Cued and 
to-be-recalled material came either from the same sentence or from adjacent 
sentences. Consistent with the theory of real-world memory under 
Investigation, between-sentence retrieval tended to Lake longer than did 
wi thin-sentence retrieval. A second experiment revealed that both retrieval 
success and retrieval speed Increases with the number of words contained in 
the cue segment. 

Poten tial Applications ; Many technical jobs in the military require similar 
real-world memory sklls. This research should be useful in improving the 
efficiency of training for these jobs as well as in evaluation of the results 
of such training. 

Reports ; . 

Ericsson, K.A. and Poison, P. G., (1985). A Cognitive Analysis of ^^^^P^^g"^;^ 
Memorv for Restaurant 0.rder_s_ ^Technical Report No. 140). University of Colorado, 



Psychology Department, 1985. 
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STRUCTURAL UNDERSTANDING IN PROBLEM SOLVING: SPECIFYING AND INSTRIIfTTNC 
COGNITIVE OBJECTIVES IN BASIC ELECTRICITY INSTRUCTING 
NR 667-538 

University of California, San Diego/Norman & Riley 

r!«.Hrp!^^' r^^-^^^^; °l'J"tive8 are to analyze (1) the knowledge 

^f differlnt^n^?"'.'. ?"'Tf " °" Problems. (2) the influence 

of different instructional methods on students' ability to acquire this 

^IZlltT'r ^"'^/^^^J^ relationship between this knowledge and the knowledge 
cnnce«t^ i / problems. A major issue concerns how general 

concepts lead to important gains in understanding at the outset of learning a 
new aomain* ° 

M2E2ach: The approach involves a combination of theoretical analyses and 

J^' ^"^ly^^s involve the development of computer 

models of the knowledge underlying performance in solving circuit problems. 
Development of the models is guided by information obtained from P""""^""- 
thinking-aloud protocols obtained from students solving problems. Protocol 
analysis is the most effective available methodology for developing detailed 
analyses of students' performance on a variety of problem-solving tasks. 

ll°XZV ^"''n^p' '° Identify the knowledge required for successful 

performance on D-C circuit problems, which components of knowledge are 
available to students at the outset of instruction, and how this knowledge 
ttT^ instruction. Work has also begun to analyze how students car. 

xeam domain-specific concepts about D-C circuits by analogy with more general 
concepts that they already have available. In addition. wSk has begun'to 

?n t °^ ^^^^ students' errors 

m terms of the mechanisms responsible. 

Potential Applications; This research could lead to improvements in the 
teaching of basic electricity concepts, such as those taught in the Navy' s 
Basic Electricity and Electronics course. 

Reports ; 

Riley, M. (1985). User Understanding (Techuical Report, ICS Peport V8504) 
University of California, San Diego, May 1985. (AD 157155). 



mS^^J^^!o/^°^^^^^^ LEARNING PROBLEM SOLVING SKILLS FROM EXAMPLES 
NR 00 7-539 

University of Pittsburgh/Chi & Glaser 



Scientific Obiectives; The goal of this research is to understand how 

i^^]/ textbooks. We also attempt to understand the sources of 

as Jb sic learning to solve problems in a technical domain such 
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Approach ; The approach used In this research Is to collect elaboration 
protocols of students studying worked-out examples and relate the kind of 
inference rules they build during the studying phase with the production rules 
that they use to solve the problems. Students' elaboration protocols are 
collected while they study worked-out examples and justify each problem- 
solving step. In contrast to this investigation of study protocols, 
conventional problem-solving research has focused predominantly on problem- 
solving protocols themselves, thus the data generally provide a competence 
model rather than a learning model. 

Progress: We have completed the collection of our data on 10 students 
learning four chapters of physics text, their protocols during the studying 
phase, and their protocols for solving problems that are isomorphic to 
the example problems as well as problems that are quite distinct from the 
examples. 

Pote ntial Applications ! This work should provide practical advice on how to 
design instructional materials to be more helpful to a wider range of 
students, particularly in scientific and technical domains. 

Archival Publications : 

Glaser, R. (1985). Understanding human intelligence (Review of 
Beyond IQ) , Science , 230 , 59-61, October. 

Glaser, R. and Rabinowitz R. (1985). Cognitive structure and process in 
highly competent performance. In F. D. Horowitz & M. O'Brien (Eds.), 
The gifted and the talented! A de velopmental perspective. Washington, DC: 
American Psychological Association. 



HUMAN UNDERSTA^iDING OF COMPLEX SYSTEMS 
NR 667-540 

Bolt Beranek & Newman, Inc. /Collins 



Scienti fic Objectives ; The project has two closely related objectives: (D 
to develop and refine a theory of how people's understanding of physical 
systems changes during learning, and (2) to develop a theory of explanation 
and to test which types of explanation are most effective in inducing 
conceptual change. 

Approach: A large experiment was conducted to investigate conceptual change 
across th ree domains: home heating systems, electrical systems, and evapor- 
ation processes. Subjects in the experiments were tutored individually, and 
were tested in a pre- and post-test design. A tutoring system will be 
developed that can present material on a topic using different explanation 
strategies. Then a series of experiments will be run comparing the effect- 
iveness of the different explanation strategies in a similar pre- and 
post-test design. 
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Progress; The experiment run during the year found that the most important 
thing that subjects acquire is an understanding about a ft/ basic mechanisms. 
For the domain of evaporation the first basic mechanism is the idea of 
molecules in motion, which is critical to understanding differences in 
temperature and pressure, how molecules escape (and return) from water, and 
howmolecaler exchange energy. A second basic mechanism is the notion of 
molecular attraction. It plays a role in the difference between liquid and 
vapor, the close packing of molecules in water, the difficulty in escaping 
from the liquid, and the aggregation of water molecules around charged 
particles. These and other basic mechanisms appear to be the critical 
concepts the students are learning. 

Potential Applications; This research is part of an important new subfield of 
cognitive psychology; the study of mental models. The increased 
understanding of mental models will impact on scientific and technical 
training both in the military and in the society at large. In particular, 
this research has strong implications for the improved design of intelligent 
automated instructional systems, by suggesting the most effective teaching 
strategies for overcoming students' prior misconceptions. 



COMPUTATION VIA DIRECT MANIPULATION 
NR 667-541 

University of California, San Diego/Noriran 

Scientific Objectives; The objective of this research effort is to understand 
a new style of computation, computation by direct manipulation. In this form 
of interaction the computational demands of systems are matched to the mental 
structures of users and thus allow concentration upon the substantive topic 
domain rather than the means of computation. The goal of this project is to 
develop a cognitive account of both the advantages and disadvantages of this 
new form of computation. To accomplish this will require a deeper 
understanding of the role of mental representations in task performance. The 
development of such a theory is the primary objective of this research. 

Approach ; There are two underlying metaphors for interaction between humans 
and machines. The traditional one has been that of a conversation in which 
the interface serves as an intermediary between a user and a woild about which 
things are said. A second metaphor, which underlies the notion of direct 
manipulations, is that of a model world. In a system built on the model- 
world metaphor, the interface is itself a world where a user can act, and that 
changes state in response to user actions. The world of interest is 
explicitly represented and there is no feeling of an intermediary between user 
and world. It is this form of interaction that we are investigating. The 
technical approach being taken consists of integrated theoretical, empirical, 
and development efforts. The development effort is primarily concerned with 
Che design and implementation of direct-manipulation programming environments 
for a series of test domains. A related activity is the development of a set 
of computational tools to facilitate the implementation of the applications. 
The empirical effort involves investigating test systems in order to 
understand the nature and role of effective mental models and to develop 
analytic techniques for the elicitation and representation of these models. 
The theoretical effort focuses on the development of the necessary 

162 



1/u 



psychological principles to provide a principled uudersf.andinij of 
direct-manipulation forms of human computer interaction. 

Progress ; This research is being conducted by a collaborative research group 
consistin g of researchers from NPRDC and UCSD under the auspices of the ONR 
Bridges program. During the first year a joint laboratory was established. 
Maior areas of progress are: Completion of a book on user-centered system 
design; development of a theoretical framework for the investigation of direct 
manipulation interfaces (Two phenomena which give rise to the feeling of 
directness of interfaces were identified. One has to do with the information 
processing distance between the user's intentions and the facilities provided 
hy the machine. The other concerns the relation between the input and output 
vocabularies of the interface languages); development of an initial implementa- 
tion of a direct-manipulation interface in the area of statistics. 

Report : 

Hutchins. E.. Hollan. J.D.. & Norman. D.A. (1985). Direct manipulation 
interfaces . (Technical Report ICS No. 8503). University of California, 
San Diego,' Institute for Cognitive Science. 

Archival Publication ; 

Hollan, J. D., Weitzman, L, & Rosenstein, M. (1985). Interface design for 
simulations. In Proceedings of the First Annual Artificial 
Intelligence & Advanced Computer Technology Conference, 357-Jb«. 



LEARNING ABOUT EQUIPMENT FROM TECHNICAL DOCUMENTATION 
NR 667-543 

University of Michigar/Kieras 

Scientific objectives: The work focuses on how documentation specifies both 
procedural knowledge (information about how to operate the equipment) and also 
how-it-works knowledge (information about how the equipment works inside, 
which corresponds to providing mental model information about the equipment). 

Approach; The proiect will use a combination of experimental data ard 
cognitiv e simulation modeling. The acquisition of procedural knowledge from 
technical text will be investigated by careful manipulations of the structure 
and content of instructions to operate simple equipment. Simulation models 
will be constructed to represent how instructional text is translated into a 
correctly executing set of production rules. The model should account for 
features of the instructions that affect the difficulty of this translation 
process. The role of transfer of knowledge of previously learned procedures 
in this process will also be investigated. The nature of effective mental 
models for equipment will be investigated by the systematic manipulation of 
the specific krr^'ledge supplied to tho learner about how the" equipment 
operates. Aga.n, a cognitive simulation model should be able tc account tor 
which information is critical. 
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Progress; Progress to date has consisted primarily of setting up new 
laboratory facilities for data collection, and implementing basic tools for 
doing the cognitive simulation modeling. Data collection and modeling efforts 
are about to begin. 

Potential applications; These results will be considerable value in 
specifying the form and content of technical documentation for equipment. 

Reports ; 

text''*A°nrn^ ^^^^^^ ' ^he Acquisition of procedures fron. 

lext. A production-system ana lysis of trTnsfer of training (TR-nWRTTM 

University of Michigan, Department of Engineering, January 1985. 

Kieras, David, F (1985) The role of pri or knowledge In operating eoulnm .nf from 
written instructions (T R-ONK-19). University of Michigan. Department ^of 
Engineering. February 1985. 



GENERATIVE PROCESSES IN REPRESENTATIONS OF PROBLEMS (II) 
NR 667-544 ^ 

University of California. Berkeley/Greeno 

Scientific Obiectives ; The goal of this project is to characterize knowledge 
that supports the generation of representations in problem situations. To 
understand how different individuals construct different epresentations of 
the same problem, or how an individual constructs different representations in 
JSr 667-534!) °" ^ problem, we need a generative theory. (See also 

^PP"°^^f : ^ fi^^^^"^ °^ study of problem formulation is the 

resuUs Z'^ l^'Tr T' ' '"^e body of experimental 

results. One method to be used is the construction of computational models 
that simulate general features of these existing findings, with special 

p J**!,'^"'"'^^^?^ proeesses that can generate reformations of the 
problems. Relations will be considered between the cognitive processes in 
insight problems and other tasks, especially involving knowledge about 
physical devices, such as troubleshooting and learning operating procedures, 
and for predicting and explaining events in physical systems that often reveal 
misconceptions about physical principles. New tasks will be designed to 
enable assessment of knowledge required for a variety of tasks involving the 
same physical device. ° 

Progress: A model of the knowledge needed for insight tasks has been 
developed. Involving knowledge of methods associated with goals, functions 
associated with objects, and properties of functions that can be used, if 
necessarv. to infer potential functions of objects. These knowledge 

^.^^T"^ significantly to principles of understanding physical 
tlinr i principles that generate requirements, and technoiogical 

principles that generate capabilities. This scheme is being used to design a 
task environment that can he used to identify knowledge structures needed for 
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varied problems such as learning operating procedures, troubleshooting, and 
predicting and explaining behavior of the system. In another domain, models 
of knowledge for representing problems in elementnry programming have been 
formulated, clarifying properties of knowledgp in the technical domain of 
computation that are different from knowledge brought from general experience 
by novices. 

Potential Applications ; A better understanding of knowledge required for 
representing problems (distinct from knowledge for solving problems that are 
already represented) will add to the general fund of cognitive principles that 
can provide significant guidance for design of instructional materials and for 
training individuals to perform the various tasks that are required in 
technical service. Specifically, understanding of relations between different 
kinds of tasks should assist in the design of tasks used in training when the 
goals of training include a variety of tasks to be performed in the field. 



RESTRUCTURING AND INSIGHT IN PROBLEM SOLVING 
NR 667-545 

University of Pittsburgh/Ohlsson & Resnick 



Scientific Obiectives : The goal of this project is to integrate two spcmmgly 
contradictory theories of thinking: the Gestalt view and the information- 
processing theory of problem solving. The first defines problem solving in 
terms of a sudden reorganization of the problem leading to insight, while 
the second emphasizes gradual, step-by-step progress toward problem solution. 
A resolution to the theoretical disparity has been achieved, however, by 
interpreting the Gestalt concepts of restructuring and insight in information- 
processing terms. A computer simulation model is to be developed to test 
and verify theoretical findings. 

Approach ; A small number of participants are studied intensively while 
solving classical Gestalt insight task? . Verbal protocols are obtained using 
the "think-aloud" method. These data are used to develop a simulation program 
which takes a problem description as input and provides a problem solution. 
Testing the 3imulation program with problems from a variety of task domains 
will reveal whether the mechanisms postulated in the integrated theory are 
sufficient to produce insightful behavior. 

Progress; The simulation model has been designed and is currently being 
implemented. The program is being debugged by running it on a set of very 
simple problems, such as classical problem-solving puzzles, transitive 
inferences, blocks-world problems, etc. In order to be able to solve the 
classical Gestalt insight problems, a more realistic knowledge base is 
required; this is ourrently being constructed. 

P otential Application ; The project promises to provide greater conceptual 
unity to the psychology of thinking by elaborating significant convergences 
between two major approaches to problem solving. The results will contribute 
to our understanding of the processes involved in problem solving in novel 
problem situations. 
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PARALLEL niSTRIBUTED PROCESSTNr MODK? S OF PTCflFP COCNTTTVn: PROCESSES 
NR 667-548 

Uriversity of California, San nieRo/Rumelh,irt 

Oblecttves : The pc.-u is to develop the concept of the mental model 
within the frarcvork of ponOlel distributed processing theorv. The idea is 
to conceptualise the highly cognitive notion of mental models" in terms that 
will yield accounts that are biologically plausible and computationally 
sufficient. ^ 

Approach ; The parallel distributed processing approach to understanding 
perceptual and cognitive phenomena is based on the idea that intelligent 
function manifests the activity of large numbers of highly interconnected 
processing elements possessing neuron-like properties of activation and 
connectivity. This idea is used in an effort to develop a ccirputer simulation 
capturing important features of the notion of rental models as these might 
function to guide and tune complex coordinated motor activity. A kev concept 
of this effort is the notion of a neural network which takes as input some 
specifications of the motor actions one intends to carry out and produces an 
Interpretation of "what would happen if one did that." Part of this 
specification is a specification of what the new stimulus conditions would be 
once the anticipated action is completed. If the anticipated new stimulus 
conditions are inconsistent with a desired state of affairs, then some 
alternative action set would be considered. This iterative process would 
continue until a match between desired and anticipated states of affairs is 
achieved. 

Progress ; A parallel distributed processing model exhibiting the features 
outlined above has been developed, and is currently being used as the basis 
for a computer simulation. This simulation will serve to assess the 
computational and psychological adequacy of the model. 

Potential Applications; The model under development in this research effort 
has strong potential applicability in the design of robotic systems that 
possess adaptive flexibility in response to changing work environments. 

Reports ; 

Rumelhart, D,, Hlnton, G. E. and Williams, R.J. (1985). Learning interna l 
representations by error propagation (Technical Report ICS No. 8506) 
University of California, San Diego, Institute for Cognitive Science! 
September 1985, 



Rumelhart, D.E. and McClelland, J.L. (1985). On learning the past tenses of 

en£Hsh_verbs (Technical Report ICS No. 8507). University of California, 

San Diego, Institute for Cognitive Science, October 1985. 
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AhALOGTCAL PROCFSnTNG AND EXPFPTFNTIAL TT.ARNING IN PHYSICAL DOMAINS 
NR 667-551 

University of T ] ] irois/r.fintner 

Scientific Obiective'^ : Cor^t^nrt nT>d test a general theory of analogical 

prorc'ssing fip it applies to le^irning nml reasoning, and develop a 

computet 'onal account of bow people access and process analogy and similarity. 

Tccbr.ical Approac h: Centner's Structure-napping tbeory forms the basis for 
this research. Resenrrb techniques derive from both psychology and 
artificial intelligence. Thev include: (1) psychological experiments to 
investigate the processing of analogy and similarity; (2) computer simulation 
of analogical processes; (3) analyses of people's understanding of systems and 
of the changes that occur in understanding as a result of different 
experiences. 

Progress: Work undar this contract was begun late in FY-85 and no progress 
reports have yet been issued. However, this effort Is a continuation of work 
done under an earlier sub-contract, and the publications below reflect that 
earlier work. 



Pot ential Applications : By illuminating how people learn and use mental 
models of systems they must deal with, this research will contribute to b 
system design, to improved training, and to enhance maintenance ability. 



etter 



Archival Publications ; 

Centner, D. & Grudin, J. (1985). The evolution of mental metaphors in ^ 
psychology; A ninety-year retrospective. American Psychologist, 40, No. ^, 
181-192. 

Forbus, K., & Gentner, D. (1985). Learning physical domains; Towards a 
theoretical framework. In R. M. Michalski, J. Carbonell 6 T. Mitchell (Eds.), 
Machine learning ; An artificial intellige nce approach. Vol. JI» Morgan 
Kaufmann. 

Forbus, K. (1985). Qualitative process theory. Artificial Intelligence, 24. 
Also published as a chapter in D. Bobrow (ed.) Qualitative reasoning about 
physical systems . Cambridge, MA; MIT press. 

Forbus, K. (1985). The role of qualitative dynamics in naive physics. In J. 
Hobbs 6 R. Moore (Eds.), Formal models of the co mmonsense world. Tioga Press. 



THE EFFECTS OF GRADUATE TRAINING ON INDUCTIVE REASONING 
NR 667-552 
University of Michigan/Nisbett 

Scient ific Objectives ; The objective is to determine the effects of various 
kinds of graduate training on indu-cive and deductive reasoning. The 
anticipation was that the "probabilistic sciences" would improve inductive 
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reasoning and deductive reasoning whereas nonscience fields and deterministic 
science fields would Improve deductive but not Inductive reasoning. 

Approach ; A test consisting of items assessing statistical and other types of 
inductive reasoning as well as items assessing logical and formal reasoning 
was administered to students just embarking on graduate education and to 
students who had studied for two years. The fields examined, all at the 
University of Michigan, were chemistry, psychology, law and medicine. 

Progress; As anticipated, training in both of the probabillstir fields, 
psychology and medicine, produced substantial improvement in Inductive and 
deductive reasoning. These results can be understood if we assume that the 
probabilistic sciences teach principles of Inductive reasoning that are 
generally applicable, whereas nonscience fields and deterministic science 
tlelds do not teach people how to deal with uncertainty. All fields studied, 
however, teach people how to analyze problems in a formal way. consistent with 
logical requirements. A longitudinal version of the study is currently under 
way. The original first-year students have now completed their second year of 
study and have been retested. It will soon be possible to see whether the 
longitudinal results are the same as the cross-sectional results. 

Potential Applications; It is possible that the inductive reasoning 
principles taught by fields such as psychology and medicine can be 
considerably encapsulated and taught effectively with techniques less 
Intensive than two years' limnerslon in psychology or medicine. It is also 
possible that even the benefits conveyed by such immersion can be considerably 
enhanced by appropriate additional instruction. 

Archival Publication ; 

Nlsbett, R. E. b Kunde, A. (1985). The perception of social distributions. 
Journal o f Personality and Social Psychology . 48. 297-311. 



EXPLANATION AND LEARNING IN PROCEDURAL SKILLS 
NR 702-009 

University of Colorado/Lewis 

Scientific Ob.lectlves; To understand how people use causal analysis In 
learning complex skills, especially skills in controlling computer systems. To 
use this understanding to design systems that are easier to analyze and hence 
easier to learn to control. 

Approach : The project uses three converging approaches. First, empirical 
study of human/ learners is used to document the role of causal analysis, as 
revealed by learner's behavior and verbal comments. Second, a model of the 
analysis process, and how its results can be used to drive skilled 
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performance, is implemented in the form of a computer program. The model is 
based in part on observations of human learners, and in part on rational 
analysis of the learning task. It serves as a statement of the evolving 
theory as well as providing guidance for continuing empirical study. Third, 
tests of the practical implications cf the theory for the design of systems 
and the design of training methods are planned. 

Progress ; An initial learning model for user-computer interactions has been 
built, embodying three simple heuristics for identifying causal relationships. 
They are: identity, loose ends, and previous action. Surprisingly, these 
heuristics are adequate to analyze a number of common interaction patterns, 
without requiring any other knowledge. While it is clear that human learners 
often rely on specific knowledge of computer systems in learning new 
interactions, it is nonetheless interesting that this specific knowledge may 
often be logically unnecessary. Even in its present rough form the model can 
deal with some cases that are beyond the reach of approaches that do not use 
causal analysis, such as methods that modify interactions by replacing 
references to one action or object by another. 

Potential Applications ; The project may help us to distinguish clearly 
between user-computer interactions that can readily be understood and learned 
and those that require complex problem-solving or prior knowledge to master. 
It may also suggest how to design new forms of interactions that exploit our 
insight about what learners must do in mastering a new interaction. Beyond 
the sphere of user-computer interactions it is likely that causal analyses are 
involved in other skills, especially other control skills such as are involved 
in using any complex equipment, and that design improvements in other areas 
might be forthcoming. Even when interaction designs cannot be modified the 
project may be useful in suggesting how a skill could best be presented to a 
learner. It is possible that commentary that highlights the causal 
connections between user actions and system responses may enhance learning 
from demonstrations, and the project will explore this possibility. 



TRANSFER OF TRAINING IN PROBLEM SOLVING 
NR 702-Gll 

Community College of Allegheny County/Kotovsky 

Scientific Obiectives; The goal of the research is to obtain an 
understanding of factors that determine transfer of training in problem 
solving (that is, how solving one problem facilitates the solution of 
subsequently encountered problems). A second goal is to build a computer 
model of the transfer process. 

Approach ; In the early phase of the project, a number of experimental 
investigations of transfer will be carried out. These will involve measuring 
the amount of transfer obtained between pairs (and sometimes larger numbers) 
of problems administered at varying time intervals. The problems are sets of 
isomorphs of the Tower of Hanoi problem and of simple physics problems that 
will be presented to subjects on a computer-graphics system. They differ in 
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their underlying move-rule similarity. As our knowledge accumulates, a 
computer model of the transfer process will be constructed to explore the 
interactions of the factors found to have a significant effect on transfer. 

Progress ; Work has begun on programming sets of isomorphic problems, and on 
developing the stimulus presentation and data collection methods. A basic 
stimulus presentation-data collection system is currently near ccs-nletion. 
The first series of experiments is about to begin. 

Potential Applicationsu ; This work should provide guidance or the design of 
many instructional sequences. It should provide information that can be used 
to determine the makeup and sequencing of materials so as to increase the 
opportunities for problem-to-problem transfer, and thus for the eventual 
generalization of problem-solving skills. 

Archival Publication ; 

Kotovsky. K., Hayes, J. R., 6 Simon, H. A. (1985). Why are some problems 
hard? Evidence from Tower of Hanoi. Cognitive Psychology . j_7, 248-294. 



CALIBRATION OF COMPREHENSION 
NR 702-012 

University of Wisconsin/Glenberg & Epstein 

Scientific Obiectives: When readers attempt to learn from written texts they 
face two problems. The first is the problem of comprehensi jn~how to engage 
the appropriate cognitive processes to extract information from the text. The 
second problem is one of meta-comprehersion~how to determine if the 
comprehension processes have succeeded. Our research on the meta- 
compreheusion problem has demonstrated poor calibration of expository texts; 
Confidence in comprehension is poorly correlated with performance on a test 
of comprehension. The current research has two objectives. The first is 
to understand the reasons for poor calibration of comprehension. The second 
objective is devising training procedures to improve calibration of 
comprehension. 

Approach; In our standard experimental procedure subjects are asked to read 
expository texts on different topics. Each text is one paragraph long and is 
composed around a central principle. Subject, are asked to judge (for each 
text) their confidence in ability to apply the principle. Finally, a test 
requiring application of the principle is given. The standard dependent 
variables are the confidence judgments, performance on the test, and a 
measure of association between the judgments and performance. These 
dependent measures are supplemented by others, such as reading time, when 
demanded by the experimental logic. Data analysis proceeds through a 
combination of ANOVA and multiple regression. The regression analyses are 
useful when dealing with individual-difference variables measured on 
continuous scales. 
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Progress ; Two findings, based on prelipinnry research, appear significant. 
First, in the absence of specific fccr'back, subjects are poor at predicting 
performance on efl types of comprehension tests. However, experience in 
answerirg questions specific to a givei^ text enhanccj; predictions of future 
performance. Second, experts in a given domain are no better calibrated ir 
that domain thai: arc novices. Internal analyses of the data suggest that 
experts predict future performance on the basis of their self-classification 
as experts, not by assessing knowledge acquired from specific fexts. 

Potential Applications ; The application., of this work are not limited to the 
conventional student population. Thete are many non-school settings in which 
people are required to engage in frequent updat^fng to keep pace with 
developing technologies or shifting missions. In these cases, awareness of 
the need to train self-monitoring and to improve calibration may contribute to 
less strfssful and more efficient operations. 



COMPREHENSION PROCESSES IN MECHANI REASONING 
NR 702-014 

Carnegie-Mellon University/ Just & Carpenter 

Scie ntif-lc Objectives : This research aims to analyze the process by which 
"TTeader of texts describing mechanical systems constructs ar internal 
representation of the mechanisms in order to operate, repair or assemble the 
mechanism later on. Further, the research aims to determine how the 
properties of texts and diagrams, as well as the cognitive abilities of the 
reader, affect this process. 

Approach ; Modified versions of texts from B?sic Machines , a Navy training 
manual, will be used in the research. Characteristics of the text, thp 
diagrams, and the relation between the two, will be experimentally v-aried. 
Records of eye movements by readers will provxde data for making inferences 
about processing, particularly identification of points of maximum processing 
load. A computer simulation will be produced that builds up a"mental model" 
of the mechanism. This simulation will be based upon existing simulations of 
text processing and of the processing o^ patial ability test items. 

Progress ; This is a new contract, and no : tatus Reports have yet been issued. 

Potential Application s; This research should le.^d to methods (possiblv 
automated) and guidelines for both the effective use of cxagrams in relation 
to text and for determining whether procedural instructions contain sufficle-t 
and appropriate information to petr^it adequate understanding. 



EFFECTS OF SECOND-LANGUAGE PPOl CIENCY ON LEARNING FPOM TEXTS 
NR 702-015 

Univ^i -ty of California, Santa Barbara/Duran & Goldman 

Scle'-' '^ ^cObiectives ; The overall objective is to characterize the cognitive 
procer^js of linguistic minority trainees as they comprehend and reason from 
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instructional texts, l.iis includes the Identification of linguistic 
structures that are likely to cause difficulty, as well ae the investigation 
of the impacf of lower-level aspects ot comprehension upon nore sophisticated 
inferentl^^l processing of Instructional texts. 

Approach; Texts from a variety of subject-matter areas are beting analyzed 
with respect to structure? nt the sentence, Inter-sentence, and global level. 
Quantifiers, qualifiers, conditionals, and linguistic features signaling 
cohesion among parts of a text are being emphasized. Students are presented 
with academic tasks varying in their demand for sophisticated cognitive 

rocesslng, and are asked to "think aloud" as they perform the tasks in order 
to Identify points of processing difficulty. Exploratory studies during the 
early phase of the contract will lead to hypotheses to be tested 
experimentally in later phases. 

Progress ; This is a new contract. 

Potential Applications; Members of linguistic minorities, primarily Hispanic 
Americans, constitute a large and increasing proportion of potential naval 
pe.sonnel. This research is aimed at understanding how limitations of 
linguistic proficiency — even among personnel who seem relatively proficient 
~ can interfe/e with thorough comprehension of training materials and prevent 
success in technical training. Guidelines tor appropriate design of training 
materials are one expected payoff of this research. 



****** 
THE JLLO.-.'I.It WORK UNITS WERE COMPLETED DURING THE PAST YEAR: 

EVALUATION OF THE ABILITIES OF MARINE ENGINEERING PERSONNEL TO DETECT 
DIAGNOSE, AND COMPENSATE FOR SYSTEM FAILURES 
NR 154-491 

Georgia Institute of Technology/Rouse 

Scientific Obiectives: The goal was to Investigate human problem-solving 
behavicr and its relation to simul.nor fidelity. A principal objective was to 
determine the levels of fidelity most appropriate for learning the cognitive 
skills characteristic of expe -t troubleshooters . 

Approach: Marine engineering personnel in training at Marine Safety 
International, Inc., were studied - they detected, diagnosed, and responded 
to system failures in a high-fidell / simulation of a supertanker powerplant. 
Behavic r was also studied in the context of computer-based low and moderate- 
fidelity simulators. The relations among performance with simulators at 
different fidelity levels was analyzed. Measures of problem-solvirg 
performance, system knowledge, and individual ability were taken. Results 
were used to refine a rule-based model of diagnostic reasoning and problem 
solving. 

Results a nd Conclusions ; Fidelity can be characterized along physical, 
structural, and dynamic dimensions. Pattern recognition, graph-theoretic, and 
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control-theoretic methods, respectively, ara appropriate for > oping 
quantitative measures of fidelity along each of these dimensions. Research on 
modeling engineered systejis and building intelligent simulator-based training 
systems can benefit from a combination of qualitative reasoning methods and 
exact methods based on differential equations. A hybrid language of formalism 
for simulating large dynamic systems is needed also for the development of 
intelligent aids to real-time problem solving. Accurate expert performance in 
dia'nostic tasks depends upon a combination of task-specific system knowledge 
an'i 'generic problem solving strategies. Individual differences in both these 
areas can be measured and related to differences in task performance. 

Potential Applications ; Marine powerplant failures which cannot be handled by 
automated subsystems are usually complex. They require highly developed 
skills for timely detection, diagnosis, and compensation. Simulators are 
central to effective training, yet the cost of high-fidelity simulators limits 
the number of personnel who can be effectively trained. This research has 
broad applications to improving the cost-effectiveness and availability of 
simulator-based training by providing scientifically based design principles ^ 
for less expensive simulators which can be Implemented on microcomputers or Aj. 
work stations. 



Reports ; 

Rouse, W. B., b Morris. N. M. (1965). O n looking i nto the black box; 
Prospects and limits in the search for mental models (Tech. Rep. No. SIj-Z) . 
Atlanta, GA: Georgia Institute of Technology, Department of Psychology. 

Su P.-L. D. (1985). Modeling fault diagnosis performance on a marine 
powerplant simulator (Tech. Rep. No. 85-3). Atlanta, GA: Georgia Institute 
of Technology, Department of Psychology. 



Archival Publication ; 

Govindaraj, T., Ward. S. L.. Poturalski, R. J., & Vikmanis, M. M. (1985). An 
experiment and a model for the human operator in a time-constrained competing 
task environment. IEEE Transactions on Systems, Ma>i . and Cybernetics, SMC- 15 
496-503. 



INTELLIGENT INSTRUCTIONAL METHODS FOR TEACHING PROCEDURAL SKILLS 
NR 154-493 

BBN Laboratories/Feurzelg & FrederlckLon 



Scientific Objectives ; The objective was to test a model of the knowledge and 
reasoning processes used by high-level expert electrical system 
troubleshooters, and to use this model as the basis for the design of an 
Instructional system for ter..^hing qualitative and quantitative problem-solving 
skills. 
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A££r2ach: Detailed studies of expert electrical system troubleshooters were 
carried out. A model of expert troubleshooting and a qualitative causal model 
of circuit operation were developed and refined. These models were used to 
demonstrate correct diagnostic procedures to students and to -vpiain -he 
operation of circuits in faulted and unfaulted conditions. 

Results and Conclusions; This project res-alteo in the formulation and 
implementation of an instructional approach based on a carefully designed 
sequence of qualitative causal knowUdge structures, or "mental models " Each 
mental model in the sequence is adequate to solve a well-defined set of 
problems. Thus, the sequence represents an explicit characterization of the 
earliest levels of novice understanding. Intermediate levels of systematic but 
incomplete understanding, and advanced levels of expertise. The instructional 
system also incorporates a simulated expert system, which can solve circuit 
problems while explaining its actions graphically and verbally (by voice 
synthesis). These results demonstrate the objective characterization of 
mental models at different levels of understanding, as well as the utility of 
such models in the design of instructional systems. 

Potential ApplJcaticn^; The computer-based instructional methods and 
qualitative causal modeling techniques could have wide applicability for 
training complex procedural and diagnostic skills. Further, the particular 
approach to training electrical troubleshooting appears to be relatively easy 
to l^arn, and to be compatible with existing courses on electrical systems and 
their maintenance. 

Re ports ; 

White, B. Y.& Frederickson, J.R. (1985). QUEST; Qualitative understanding 
ot electrical system t roubleshooting . Cambridge, MA; BBN Laboratories. 

IZtl'l^'^T-''' A system for teaching a qualitative 

understanding of electrical circuit beha vior and troubleshoot^n^ r.n.K.-,Hf.» , 
i-IA; BiJN Laboratories. 



THE ROLE OF PRIOR KNOWLEDGE iN OPERATING EQUIPMENT FROM WRITTEN INSTRUCTIONS 
NR 667-473 

University of Arizona/Kieras 

Scientific Obiectives; This project investigated the role of three kinds 
of prior knowledge: previous knowledge of equipment in general, such a^; 
electronics experts would have; prio- knowledge of how the equipment work, 
inside, as in a mental model for the equipment; end prior knowledge of sone of 
the operating procedures. 

A£'.roach: Electronics experts were compared to ordinary subjects in ta^'ks 
involving describing equipment, or operating it according to instruction 3. 
Subjects who knew a mental mode] for a fictitious device wr^re compared with 
those who didn't in terms of their ability to learn, or Infer, how to operate 
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the device. Prior knowledge of procedures was studied by careful manipulations 
of the order in which subjects learn procedures. In all three topic areas, a 
key approach was to develop a cognitive simulation model of the processes 
subjects are performing, and then to compare the model to the behavioral data. 

res ults and Conclusions : The major results can be summarized as follows: 
Knowledge of equipment in general is very complex, but does not seen to 
include many stereotyped procedures. Experts can be misled b> their knowledge 
when dealing with unfamiliar equipment in instruction-following situations. A 
mental model for a device is valuable only if it enables the user to infer the 
exact procedures for operating the device. A simulation model successfully 
represents this process. Transfer of training trom previously learned 
procedures can be predicted very accurately based on a formal description of 
the procedural knowledge required. 

Potenti al Applications ; The results of expertise in instruction-following are 
important for the design of computerized maintenance aids. The mental model 
ard transfer results suggests possible improvements in training and reference 
materials. 

Reports ; 

Kieras. D.. & Bovair. S. (1985). The acquisit ion of procedures from text: 
A production-system analysis of transfe r of training (Technical Report No. 
Te)"; University of Michigan. Department of Psychology . 

Floras. D. E. (198b). The role of prior knowledge in operating eguipmen c 
from written instructions ; Final Report (Technical Report No. 19). 
University of Michigan. Department of Psychology. 

Archival Pub] i cation ; 

Kieras. D. E. (1985). The why. when, and how of cognitive simi-lation: A 
tutorial. Behavior Research Methods Instruments. & Computers. 17' ^/^-^tt^- 



EXPERT PROGRAMMER COMPREHENSION OF REALISTIC COMPUTER PR0G»;il1S 
NR 667-503 

University of Chicago/Pennington 

'Scientific Objectives ; The goal was to test and refine a theory of ^^^pertise 
and knowledge representation in the domain of computer programming. This 
project focused on experienced, expert progiammers rather than novices. 

Approach: Exploratory and experimertal methods were used to study the 
comprehension of and memory for computer programs. Moderately long, 
realistic programs were constructed in FORTRAN nnd COBOL. These programs 
were then used in think-aloud protocol, free recall, ard cued reca., 
experiments. Recall and recognition accuracy, inter-response times, arc 
conceptual clustering were measured. Tlii« r.ultiva'-iate approach allowed 
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cross-checking, of hypotheses about memory structures underlying expert 
performance. 

Results and Conclusions; Computer programming and other design tasks can be 
analyzed m terms of multiple abstractions which interact to form the 
meaning of the designed structure. Thus, comprehension of the final form of 
a program required retrieval and integration of multiple sets of relations 
among parts. Experiments tested which, if any, representations are dominant 
in the comprehension of programs by experts; specifically whether procedural 
(control flow) or functional (goal hierarchy) relations predominate. Results 
of these experiments agree with other data from studies of text comprehension: 
procedural units are most salient. 

Potential Applications; Programming language and software systems designers 
should benefit from a better understanding of the ways programmers think. 
Improvement in le guage design, educational practices, and programming aids 
are needed to overcome the current software crisis, which is very costly to 
the military and to industry. 

Report ; 

Pennington. N. (1985). Stimulus structures «nri mental representat ions in 
expert comprehension of computer programs (Tech. Rep. No. 2-ONR) . University 
or Cnina^o. ^ 
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